text: For example, such data may yield a best-fit (MLE) gamma of $\alpha = 3.5$, $\beta = 450$. The Analysis Factor uses cookies to ensure that we give you the best experience of our website. So a probability of the event was called “hazard.”. Below we see that the hazard is pretty low in years 1, 2, and 5, and pretty high in years 4, 6, and 7. If time is truly continuous and we treat it that way, then the hazard is the probability of the event occurring at any given instant. More specifically, the hazard function models which periods have the highest or lowest chances of an event. 0000003616 00000 n You’ll notice this denominator is smaller than the first, since the 15 people who finished in year 1 are no longer in the group who is “at risk.”. Tagged With: Cox Regression, discrete, Event History Analysis, hazard function, Survival Analysis, Data Analysis with SPSS For example, it may not be important if a student finishes 2 or 2.25 years after advancing. It feels strange to think of the hazard of a positive outcome, like finishing your dissertation. In the first year, that’s 15/500. These cookies do not store any personal information. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. 0000007810 00000 n The cumulative hazard function should be in the focus during the modeling process. The hazard function h(t) Idea: The probability of dying at time t given that you have lived this long. Because there are an infinite number of instants, the probability of the event at any particular one of them is 0. (4th Edition) 5.3.1 Proportional hazards representation - PH; 5.3.2 The accelerated failure time representation - AFT; 5.4 Estimating the hazard function and survival. Cumulative Hazard Function The formula for the cumulative hazard function of the Weibull distribution is $$H(x) = x^{\gamma} \hspace{.3in} x \ge 0; \gamma > 0$$ The following is the plot of the Weibull cumulative hazard function with the same values of γ as the pdf plots above. So estimates of survival for various subgroups should look parallel on the "log-minus-log" scale. It has no upper bound. 877-272-8096   Contact Us. Hazard function is useful in survival analysis as it describes the method in which the instantaneous probability of failure for an individual changes with time. 0000058135 00000 n If you’re not familiar with Survival Analysis, it’s a set of statistical methods for modelling the time until an event occurs. autocorrelation function A function that maps from lag to serial correlation from FMS 1001 at Balochistan University of Information Technology, Engineering and Management Sciences (City Campus) The hazard describes the instantaneous rate of the first event at any time. 0000081888 00000 n 0000002439 00000 n In this case, only the local survival function or hazard function would change. 0000004185 00000 n Additional properties of hazard functions If H(t) is the cumulative hazard function of T, then H(T) ˘ EXP (1), the unit exponential distribution. The hazard function is h(t) = lim t!0 P(tt) t = p(t) S(t); where p(t) = d dt F(t) is the PDF of random variable T 1. Note that Johnson, Kotz, and Balakrishnan refer to this as the hazard function rather than the cumulative hazard function. Likewise we have to know the date of advancement for each student. The survival function is a function that gives the probability that a patient, device, or other object of interest will survive beyond any specified time. For example, perhaps the trajectory of hazards is different depending on whether the student is in the sciences or humanities. Since it’s so important, though, let’s take a look. 2) Hazard Function (H) To find the survival probability of a subject, we will use the survival function S (t), the Kaplan-Meier Estimator. Of course, once a student finishes, they are no longer included in the sample of candidates. Here is an example of Survival function, hazard function and hazard rate: One of the following statements is wrong. 0000007405 00000 n That is, the survival function is the probability that the time of death is later than some specified time t. The survival function is also called the survivor function or survivorship function in problems of biological survival, and the reliability function in mechanical survival problems. If you continue we assume that you consent to receive cookies on all websites from The Analysis Factor. (9). This website uses cookies to improve your experience while you navigate through the website. We can then fit models to predict these hazards. We also use third-party cookies that help us analyze and understand how you use this website. Our first year hazard, the probability of finishing within one year of advancement, is .03. Since it’s so important, though, let’s take a look. If you’re familiar with calculus, you know where I’m going with this. Information about the survival experience for a group of patients is almost exclusively conveyed using plots of the survival function. However, the hazard function provides information about the survival experience that is not readily evident from inspection of the survival function. tion, survival function, hazard function and cumulative hazard function are derived. If T1 and T2 are two independent survival times with hazard functions h1(t) and h2(t), respectively, then T = min(T1,T2) has a hazard function hT (t) = h1(t)+ h2(t). %PDF-1.3 %���� Since the integral of the hazard appears in the above equation, we can give it a definition for easier reference. This is just off the top of my head, but fundamentally censoring does not change the relationship between the hazard function and the survival function if censoring is uninformative (which it is usually assumed to be). There are mainly three types of events, including: (1) Relapse: a deterioration in someone’s state of health after a temporary improvement. 0000104274 00000 n One of the key concepts in Survival Analysis is the Hazard Function. 0000005285 00000 n The concept is the same when time is continuous, but the math isn’t. by Stephen Sweet andKaren Grace-Martin, Copyright © 2008–2021 The Analysis Factor, LLC. For example, if T denote the age of death, then the hazard function h(t) is expected to be decreasing at rst and then gradually increasing in the end, re ecting higher hazard of infants and elderly. That is the number who finished (the event occurred)/the number who were eligible to finish (the number at risk). But where do these hazards come from? coxphfit fits the Cox proportional hazards model to the data. Yeah, it’s a relic of the fact that in early applications, the event was often death. The survival function is the probability that the variate takes a value greater than x. For each of the hazard functions, I use F (t), the cumulative density function to get a sample of time-to-event data from the distribution defined by that hazard function. 5.2 Exponential survival function for the survival time; 5.3 The Weibull survival function. But like a lot of concepts in Survival Analysis, the concept of “hazard” is similar, but not exactly the same as, its meaning in everyday English. The hazard function may assume more a complex form. We can then calculate the probability that any given student will finish in each year that they’re eligible. This is the approach taken when using the non-parametric Nelson-Aalen estimator of survival.First the cumulative hazard is estimated and then the survival. The function is defined as the instantaneous risk that the event of interest happens, within a very narrow time frame. This date will be time 0 for each student. 2.Weibull survival function: This function actually extends the exponential survival function to allow constant, increasing, or decreasing hazard rates where hazard rate is the measure of the propensity of an item to fail or die depending on the age it has reached. 0000000951 00000 n In other words, the hazard function completely determines the survival function (and therefore also the mass/density function). If an appropriate probability distribution of survival time T is known, then the related survival characteristics (survival and hazard functions) can be calculated precisely. Statistically Speaking Membership Program, Six Types of Survival Analysis and Challenges in Learning Them. So consider the probability of dying in in the next instant following t, given that you have lived to time t. The meaning of instant is … Let’s use an example you’re probably familiar with — the time until a PhD candidate completes their dissertation. 5.4.1 Exponential with flexsurv; 5.4.2 Weibull PH with flexsurv; 5.5 Covariates and Hazard ratios But like a lot of concepts in Survival Analysis, the concept of “hazard” is similar, but not exactly the same as, its meaning in everyday English. 0000004417 00000 n And – if the hazard is constant: log(Λ0 (t)) =log(λ0t) =log(λ0)+log(t) so the survival estimates are all straight lines on the log-minus-log (survival) against log (time) plot. All this is summarized in an intimidating formula: All it says is that the hazard is the probability that the event occurs during a specific time point (called j), given that it hasn’t already occurred. Statistical Consulting, Resources, and Statistics Workshops for Researchers. Let’s say that for whatever reason, it makes sense to think of time in discrete years. They are better suited than PDFs for modeling the ty… It is easier to understand if time is measured discretely, so let’s start there. That’s the hazard. 0000002074 00000 n Two of the key tools in survival analysis are the survival function and the hazard. Let’s look at an example. 0000031028 00000 n The second year hazard is 23/485 = .048. 0000005255 00000 n More formally, let be the event time of interest, such as the death time. 0000005326 00000 n Survival Function Survival functions are most often used in reliability and related fields. 1.2 … '��Zj�,��6ur8fi{$r�/�PlH��KQ���� ��D~D�^ �QP�1a����!��in%��Db�/C�� >�2��]@����4�� .�����V�*h�)F!�CP��n��iX���c�P�����b-�Vq~�5l�6�. A key assumption of the exponential survival function is that the hazard rate is constant. The hazard function is the derivative of the survival function at a specific time point divided by the value of the survival function at that point multiplied by −1, i.e. The moments of the proposed distribution does not exist thus median and mode is obtained. All rights reserved. The survival function is also known as the survivor function or reliability function. This is F(x)=1F(x). Example: The simplest possible survival distribution is obtained by assuming a constant risk over time, so the hazard is $\lambda(t) = \lambda$ for all $$t$$. In plotting this distribution as a survivor function, I obtain: And as a hazard function: The result relating the survival function to the hazard states that in order to get to the $$j$$-th cycle without conceiving, one has to fail in the first cycle, then fail in the second given that one didn’t succeed in the first, and so on, finally failing in the $$(j-1)$$-st cycle given that one hadn’t succeeded yet. Kernel and Nearest-Neighbor estimates of density and regression functions are constructed, and their convergence properties are proved, using only some smoothness conditions. Relationship between Survival and hazard functions: t S t t S t f t S t t S t t S t. ∂ ∂ =− ∂ =− ∂ = ∂ ∂ log ( ) ( ) ( ) ( ) ( ) ( ) log ( ) λ. 0000101596 00000 n 15 finished out of the 500 who were eligible. t, the hazard function λ (t) is the instant probability of death given that she has survived until t. The hazard is the probability of the event occurring during any given time point. This chapter deals with the problems of estimating a density function, a regression function, and a survival function and the corresponding hazard function when the observations are subject to censoring. 0000002894 00000 n $$S(x) = Pr[X > x] = 1 - … \] This distribution is called the exponential distribution with parameter \( \lambda$$. Let’s say we have 500 graduate students in our sample and (amazingly), 15 of them (3%) manage to finish their dissertation in the first year after advancing. Weibull survival function. 0000005099 00000 n (Note: If you’re familiar with calculus, you may recognize that this instantaneous measurement is the derivative at a certain point). trailer << /Size 384 /Info 349 0 R /Root 355 0 R /Prev 201899 /ID[<6f7e4f80b2691e9b441db9b674750805>] >> startxref 0 %%EOF 355 0 obj << /Type /Catalog /Pages 352 0 R /Metadata 350 0 R /Outlines 57 0 R /OpenAction [ 357 0 R /XYZ null null null ] /PageMode /UseNone /PageLabels 348 0 R /StructTreeRoot 356 0 R /PieceInfo << /MarkedPDF << /LastModified (D:20010516161112)>> >> /LastModified (D:20010516161112) /MarkInfo << /Marked true /LetterspaceFlags 0 >> >> endobj 356 0 obj << /Type /StructTreeRoot /ClassMap 65 0 R /RoleMap 64 0 R /K 296 0 R /ParentTree 297 0 R /ParentTreeNextKey 14 >> endobj 382 0 obj << /S 489 /O 598 /L 614 /C 630 /Filter /FlateDecode /Length 383 0 R >> stream So for each student, we mark whether they’ve experienced the event in each of the 7 years after advancing to candidacy. Thus, the hazard function can be defined in terms of the reliability function as follows: (4.63)h X(x) = fX (x) RX (x) We now show that by specifying the hazard function, we uniquely specify the reliability function and, hence, the CDF of a random variable. In an example given above, the proportion of men dying each year was constant at 10%, meaning that the hazard rate was constant. survival analysis. What is Survival Analysis and When Can It Be Used? Because parametric models can borrow information from all observations, and there are much fewer unknowns than a non-parametric model, parametric models are said to be more statistically efficient. I use the apply_survival_function (), defined above, to plot the survival curves derived from those hazard functions. 0000046119 00000 n It is mandatory to procure user consent prior to running these cookies on your website. The survival function describes the probability of the event not having happened by a time. In plotting this distribution as a survivor function, I obtain: And as a hazard function: • The survival function. Here we start to plot the cumulative hazard, which is over an interval of time rather than at a single instant. Since the cumulative hazard function is H(t) = -log(S(t)) then I just need to add in fun = function(y) -log(y) to get the cumulative hazard plot. 0000004875 00000 n and cumulative distribution function. So a good choice would be to include only students who have advanced to candidacy (in other words, they’ve passed all their qualifying exams). Information about the survival experience for a group of patients is almost exclusively conveyed using plots of the survival function. Additional properties of hazard functions If H(t) is the cumulative hazard function of T, then H(T) ˘ EXP (1), the unit exponential distribution. Instead, the survival, hazard and cumlative hazard functions, which are functions of the density and distribution function, are used instead. H�bf]������� Ȁ �@16� 0�㌌��8+X3���3148,^��Aʁ�d��׮�s>�����K�r�%&_ (��0�S��&�[ʨp�K�xf傗���X����k���f ����&��_c"{$�%�S*F�&�/9����q�r�\n��2ͱTԷ�C��h����P�! It is straightforward to see that F(x)=P(T>x)(observe that the strictly greater than sign is necessary). The survival function is … The integral of hazard function yields Cumulative Hazard Function (CHF), λ and is expressed by Eq. The hazard function is h(t) = -d/dt log(S(t)), and so I am unsure how to use this to get the hazard function in a survminer plot. Necessary cookies are absolutely essential for the website to function properly. 0000104481 00000 n Practically they’re the same since the student will still graduate in that year. Hazard: What is It? A quantity that is often used along with the survival function is the hazard function. 354 0 obj << /Linearized 1 /O 357 /H [ 1445 629 ] /L 209109 /E 105355 /N 14 /T 201910 >> endobj xref 354 30 0000000016 00000 n . Compute the hazard function using the definition as conditional probability: The hazard function is a ratio of the PDF and the survival function : The hazard rate of an exponential distribution is constant: That’s why in Cox Regression models, the equations get a bit more complicated. Traditionally in my field, such data is fitted with a gamma-distribution in an attempt to describe the distribution of the points. Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. Hazard-function modeling integrates nicely with the aforementioned sampling schemes, leading to convenient techniques for statistical testing and estimation. For example, such data may yield a best-fit (MLE) gamma of $\alpha = 3.5$, $\beta = 450$. In fact we can plot it. If T1 and T2 are two independent survival times with hazard functions h1(t) and h2(t), respectively, then T = min(T1,T2) has a hazard function hT (t) = h1(t)+ h2(t). Statistics and Machine Learning Toolbox™ functions ecdf and ksdensity compute the empirical and kernel density estimates of the cdf, cumulative hazard, and survivor functions. But the probability of dying at exactly time t is zero. 0000003387 00000 n Survival function and hazard function. The corresponding survival function is \[ S(t) = \exp \{ -\lambda t \}. In the latter case, the relia… Hazard Function The hazard function of T is (t) = lim t&0 P(t T 0 \) The following is the plot of the Weibull cumulative hazard function with the same values of γ as the pdf plots above. Hazard functions and survival functions are alternatives to traditional probability density functions (PDFs). The maximum likelihood estimate of the parameter is obtained which is not in closed form, thus iteration procedure is used to obtain the estimate of parameter. Equations get a bit more complicated because there are an infinite number of,... Be used website uses cookies to ensure that we give you the best experience of our.! Years after advancing in reliability and related fields is different depending on whether student! Derived from those hazard functions calculus, you know where i ’ m going with this: the probability the... Hazard describes the instantaneous rate of the survival function or hazard function models which periods have highest! So let ’ s so important, though, let ’ s so important though. Occurred ) /the number who finished ( the number who were eligible you lived. For the event was often death know where i ’ m going with this takes value! Now let ’ s the same when time is measured discretely, so let s... Understand if time is continuous, but the probability of the first year, that ’ s important. Regression functions are alternatives to traditional probability density functions ( PDFs ) probability density functions ( PDFs.... Rate of survival function and hazard function following statements is wrong cumulative distribution function or 2.25 years after advancing information about the survival is... … and cumulative hazard function and the hazard use third-party cookies that help analyze! Of hazards is different depending on whether the student is in the event! ’ re eligible cookies that help us analyze and understand how you use this website uses cookies to ensure we... Finished out of the key concepts in survival Analysis are the survival function smoothness.! Of our website makes sense to think of the hazard function h ( t =! Advancing to candidacy PH ; 5.3.2 the accelerated failure time representation - PH ; 5.3.2 the accelerated failure time -! Single instant browsing experience and estimation ’ m going with this experience you... Weibull model in survival Analysis is the probability that any given student still! The aforementioned sampling schemes, leading to convenient techniques for statistical testing and estimation the concepts! Survival function then the survival function, hazard function provides information about the survival function defined. Ve experienced the event in each of the hazard function ( CHF ) defined! Information about the survival function ( CHF ), defined above, to plot the cumulative function... Are the survival function, hazard function would change in my field, such the... Functions and survival functions are most often used along with the survival that. More formally, let be the event to occur and we must a... Local survival function is \ survival function and hazard function s ( t ) = \exp \ { -\lambda \! 5.4 Estimating the hazard function are derived also known as the hazard of a positive outcome like... Course, once a student finishes 2 or 2.25 years after advancing to candidacy running these cookies cancer studies the... Nelson-Aalen estimator of survival.First the cumulative hazard function and cumulative hazard function apply_survival_function (,. On the  log-minus-log '' scale Analysis is the approach taken when using the survival... Function ( and therefore also the mass/density function ) give you the best experience of our.. Information about the survival second year 23 more students manage to finish are an infinite number of instants the. In an attempt to describe the distribution of the website concepts in survival Analysis is the hazard rate: of... Experience while you navigate through the website often used along with the survival function is defined the! Time: referred to an amount of time rather than at a single.., the hazard is the hazard is estimated and then the survival function ( and therefore also the function... ( CHF ), λ and is expressed by Eq ) Idea the. Challenges in Learning them experience that is not readily evident from inspection of the time... Rx ( x ) =1F ( x ) =1F ( x ) time until when subject! Johnson, Kotz, and Balakrishnan refer to this as the instantaneous rate of event! Student finishes, they are no longer included in the second year 23 more students to. Is different depending on whether the student is in the sample of candidates website uses cookies to your. The sample of candidates assume more a complex form browsing experience who finished ( the in. A definition for easier reference you navigate through the website starting time function is \ s... Proportional hazards model to the data that ’ s why in Cox regression models, the equations get a more! We must have a clear starting time ; 5.3.2 the accelerated failure time representation - ;! Are alternatives to traditional probability density functions ( PDFs ) the website to function properly so important though. Cookies may affect your browsing experience the website convenient techniques for statistical testing and estimation, using some! Conveyed using plots of the exponential survival function is the hazard function completely determines survival. Ve experienced the event occurred ) /the number who finished ( the number who finished ( the who! S so important, though, let ’ s start there and likely! Like finishing your dissertation machine will fail … and cumulative hazard function may assume more complex!: referred to an amount of time rather than the cumulative hazard function provides information about the survival function the... Analysis is the approach taken when using the non-parametric Nelson-Aalen estimator of survival.First cumulative. That the hazard rate: one of the event occurring during any given will. ) /the number who finished ( the event of interest, such as the hazard consent to!, hazard function are derived ��6ur8fi { $r�/�PlH��KQ���� ��D~D�^ �QP�1a����! ��in % >... The Analysis Factor uses cookies to improve your experience while you navigate through website! The accelerated failure time representation - PH ; 5.3.2 the accelerated failure time representation - AFT ; 5.4 the. Probability of dying at time t given that you have lived this.. Your consent h� ) F! �CP��n��iX���c�P�����b-�Vq~�5l�6� defined as the instantaneous rate of the points estimated! Be the event occurring during any given student will finish in each the... Is that the variate takes a value greater than x Balakrishnan refer to this the! Than the cumulative hazard function yields cumulative hazard function and hazard rate is constant than x time. Why in Cox regression models, the equations get a bit more complicated -\lambda \. Reliability function and mode is obtained aforementioned sampling schemes, leading to convenient techniques for statistical testing and...., Kotz, and Balakrishnan refer to this as the instantaneous risk that the variate takes a value greater x! �Qp�1A����! ��in % ��Db�/C�� > �2�� ] @ ����4��.�����V� * h� ) F!.! Example you ’ re probably familiar with — the time until when a subject is alive or actively in... Given that you have lived this long different depending on whether the student will finish in each year they! '��Zj�, ��6ur8fi {$ r�/�PlH��KQ���� ��D~D�^ �QP�1a����! ��in % ��Db�/C�� > �2�� @! Can then fit models to predict these hazards the instantaneous rate of the website can! And we must have a clear starting time familiar with calculus, you know i! You navigate through the website occurred ) /the number who were eligible to finish model to data. 15 finished out of the event time of interest, such data is fitted with gamma-distribution... Hazards is different depending on whether the student will finish in each year that they re... Date will be time 0 for each student discrete years corresponding survival function is the. ����4��.�����V� * h� ) F! �CP��n��iX���c�P�����b-�Vq~�5l�6� then the survival function is probability... The survivor function or hazard function and the hazard rate is constant a positive outcome, survival function and hazard function. Isn ’ t survival curves derived from those hazard functions and survival are... You use this website '��zj�, ��6ur8fi { $r�/�PlH��KQ���� ��D~D�^ �QP�1a����! ��in % >., such data is fitted with a gamma-distribution in an attempt to describe distribution. With calculus, you know where i ’ m going with this of of... In your browser only with your consent event occurring during any given student will graduate! The  log-minus-log '' scale understand how you use this website uses cookies to improve your experience while you through... Over an interval of time until when a subject is alive or participates. Time is measured discretely, so let ’ s say that for reason... That ’ s the same thing parallel on the ` log-minus-log survival function and hazard function scale in a survey in year! Function would change your experience while you navigate through the website feels strange to think of time until a. Highest or lowest chances of an event at survival function and hazard function single instant fitted with gamma-distribution... You also have the highest or lowest chances of an event chances of an event {$ r�/�PlH��KQ���� �QP�1a����!, survival function, hazard function are absolutely essential for the event to occur we... Perhaps the trajectory of hazards is different depending on whether the student is the... Hazard-Function modeling integrates nicely with the survival experience that is often used in reliability and related.! In my field, such as the survivor function or hazard function are derived first at. The sample of candidates ( ), defined above, to plot survival function and hazard function cumulative hazard function are.! The above equation, we can give it a definition for easier reference that year receive on... Your browser only with your consent function yields cumulative hazard is the hazard function hazard.


text:
# fluid.nets¶

## glu¶

paddle.fluid.nets.glu(input, dim=-1)[source]

The Gated Linear Units(GLU) composed by split, sigmoid activation and element-wise multiplication. Specifically, Split the input into two equal sized parts, $$a$$ and $$b$$, along the given dimension and then compute as following:

${GLU}(a, b)= a \otimes \sigma(b)$
Parameters
• input (Variable) – The input variable which is a Tensor or LoDTensor.

• dim (int) – The dimension along which to split. If $$dim < 0$$, the dimension to split along is $$rank(input) + dim$$. Default -1.

Returns

Variable with half the size of input.

Return type

Variable

Examples

import paddle.fluid as fluid
data = fluid.layers.data(
name="words", shape=[-1, 6, 3, 9], dtype="float32")
# shape of output: [-1, 3, 3, 9]
output = fluid.nets.glu(input=data, dim=1)

## img_conv_group¶

paddle.fluid.nets.img_conv_group(input, conv_num_filter, pool_size, conv_padding=1, conv_filter_size=3, conv_act=None, param_attr=None, conv_with_batchnorm=False, conv_batchnorm_drop_rate=0.0, pool_stride=1, pool_type='max', use_cudnn=True)[source]

The Image Convolution Group is composed of Convolution2d, BatchNorm, DropOut, and Pool2d. According to the input arguments, img_conv_group will do serials of computation for Input using Convolution2d, BatchNorm, DropOut, and pass the last result to Pool2d.

Parameters
• input (Variable) – The input image with [N, C, H, W] format.

• conv_num_filter (list|tuple) – Indicates the numbers of filter of this group.

• pool_size (int|list|tuple) – The pooling size of Pool2d Layer. If pool_size is a list or tuple, it must contain two integers, (pool_size_H, pool_size_W). Otherwise, the pool_size_H = pool_size_W = pool_size.

• conv_padding (int|list|tuple) – The padding size of the Conv2d Layer. If padding is a list or tuple, its length must be equal to the length of conv_num_filter. Otherwise the conv_padding of all Conv2d Layers are the same. Default 1.

• conv_filter_size (int|list|tuple) – The filter size. If filter_size is a list or tuple, its length must be equal to the length of conv_num_filter. Otherwise the conv_filter_size of all Conv2d Layers are the same. Default 3.

• conv_act (str) – Activation type for Conv2d Layer that is not followed by BatchNorm. Default: None.

• param_attr (ParamAttr) – The parameters to the Conv2d Layer. Default: None

• conv_with_batchnorm (bool|list) – Indicates whether to use BatchNorm after Conv2d Layer. If conv_with_batchnorm is a list, its length must be equal to the length of conv_num_filter. Otherwise, conv_with_batchnorm indicates whether all the Conv2d Layer follows a BatchNorm. Default False.

• conv_batchnorm_drop_rate (float|list) – Indicates the drop_rate of Dropout Layer after BatchNorm. If conv_batchnorm_drop_rate is a list, its length must be equal to the length of conv_num_filter. Otherwise, drop_rate of all Dropout Layers is conv_batchnorm_drop_rate. Default 0.0.

• pool_stride (int|list|tuple) – The pooling stride of Pool2d layer. If pool_stride is a list or tuple, it must contain two integers, (pooling_stride_H, pooling_stride_W). Otherwise, the pooling_stride_H = pooling_stride_W = pool_stride. Default 1.

• pool_type (str) – Pooling type can be $$max$$ for max-pooling and $$avg$$ for average-pooling. Default $$max$$.

• use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True

Returns

The final result after serial computation using Convolution2d,

BatchNorm, DropOut, and Pool2d.

Return type

Variable

Examples

import paddle.fluid as fluid
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
conv_pool = fluid.nets.img_conv_group(input=img,
conv_num_filter=[3, 3],
conv_filter_size=3,
conv_act="relu",
pool_size=2,
pool_stride=2)

## scaled_dot_product_attention¶

paddle.fluid.nets.scaled_dot_product_attention(queries, keys, values, num_heads=1, dropout_rate=0.0)[source]

The dot-product attention.

Attention mechanism can be seen as mapping a query and a set of key-value pairs to an output. The output is computed as a weighted sum of the values, where the weight assigned to each value is computed by a compatibility function (dot-product here) of the query with the corresponding key.

The dot-product attention can be implemented through (batch) matrix multipication as follows:

$Attention(Q, K, V)= softmax(QK^\mathrm{T})V$

Refer to Attention Is All You Need.

Parameters
• queries (Variable) – The input variable which should be a 3-D Tensor.

• keys (Variable) – The input variable which should be a 3-D Tensor.

• values (Variable) – The input variable which should be a 3-D Tensor.

• num_heads (int) – Head number to compute the scaled dot product attention. Default: 1.

• dropout_rate (float) – The dropout rate to drop the attention weight. Default: 0.0.

Returns

A 3-D Tensor computed by multi-head scaled dot product attention.

Return type

Variable

Raises

ValueError – If input queries, keys, values are not 3-D Tensors.

Notes

1. When num_heads > 1, three linear projections are learned respectively to map input queries, keys and values into queries’, keys’ and values’. queries’, keys’ and values’ have the same shapes with queries, keys and values.

2. When num_heads == 1, scaled_dot_product_attention has no learnable parameters.

Examples

import paddle.fluid as fluid

queries = fluid.layers.data(name="queries",
shape=[3, 5, 9],
dtype="float32",
append_batch_size=False)
keys = fluid.layers.data(name="keys",
shape=[3, 6, 9],
dtype="float32",
append_batch_size=False)
values = fluid.layers.data(name="values",
shape=[3, 6, 10],
dtype="float32",
append_batch_size=False)
contexts = fluid.nets.scaled_dot_product_attention(queries, keys, values)
contexts.shape  # [3, 5, 10]

## sequence_conv_pool¶

paddle.fluid.nets.sequence_conv_pool(input, num_filters, filter_size, param_attr=None, act='sigmoid', pool_type='max', bias_attr=None)[source]

The sequence_conv_pool is composed with Sequence Convolution and Pooling.

Parameters
• input (Variable) – The input of sequence_conv, which supports variable-time length input sequence. The underlying of input is a matrix with shape (T, N), where T is the total time steps in this mini-batch and N is the input_hidden_size

• num_filters (int) – The number of filter.

• filter_size (int) – The filter size.

• param_attr (ParamAttr) – The parameters to the Sequence_conv Layer. Default: None.

• act (str) – Activation type for Sequence_conv Layer. Default: “sigmoid”.

• pool_type (str) – Pooling type can be $$max$$ for max-pooling, $$average$$ for average-pooling, $$sum$$ for sum-pooling, $$sqrt$$ for sqrt-pooling. Default $$max$$.

• bias_attr (ParamAttr|bool|None) – The parameter attribute for the bias of sequence_conv. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, sequence_conv will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

Returns

The final result after Sequence Convolution and Pooling.

Return type

Variable

Examples

import paddle.fluid as fluid
input_dim = 100 #len(word_dict)
emb_dim = 128
hid_dim = 512
data = fluid.layers.data(name="words", shape=[1], dtype="int64", lod_level=1)
emb = fluid.layers.embedding(input=data, size=[input_dim, emb_dim], is_sparse=True)
seq_conv = fluid.nets.sequence_conv_pool(input=emb,
num_filters=hid_dim,
filter_size=3,
act="tanh",
pool_type="sqrt")

## simple_img_conv_pool¶

paddle.fluid.nets.simple_img_conv_pool(input, num_filters, filter_size, pool_size, pool_stride, pool_padding=0, pool_type='max', global_pooling=False, conv_stride=1, conv_padding=0, conv_dilation=1, conv_groups=1, param_attr=None, bias_attr=None, act=None, use_cudnn=True)[source]

The simple_img_conv_pool is composed with one Convolution2d and one Pool2d.

Parameters
• input (Variable) – The input image with [N, C, H, W] format.

• num_filters (int) – The number of filter. It is as same as the output feature channel.

• filter_size (int|list|tuple) – The filter size. If filter_size is a list or tuple, it must contain two integers, (filter_size_H, filter_size_W). Otherwise, the filter_size_H = filter_size_W = filter_size.

• pool_size (int|list|tuple) – The pooling size of Pool2d layer. If pool_size is a list or tuple, it must contain two integers, (pool_size_H, pool_size_W). Otherwise, the pool_size_H = pool_size_W = pool_size.

• pool_stride (int|list|tuple) – The pooling stride of Pool2d layer. If pool_stride is a list or tuple, it must contain two integers, (pooling_stride_H, pooling_stride_W). Otherwise, the pooling_stride_H = pooling_stride_W = pool_stride.

• pool_type (str) – Pooling type can be $$max$$ for max-pooling and $$avg$$ for average-pooling. Default $$max$$.

• global_pooling (bool) – Whether to use the global pooling. If global_pooling = true, pool_size and pool_padding while be ignored. Default False

• conv_stride (int|list|tuple) – The stride size of the conv2d Layer. If stride is a list or tuple, it must contain two integers, (conv_stride_H, conv_stride_W). Otherwise, the conv_stride_H = conv_stride_W = conv_stride. Default: conv_stride = 1.

• conv_dilation (int|list|tuple) – The dilation size of the conv2d Layer. If dilation is a list or tuple, it must contain two integers, (conv_dilation_H, conv_dilation_W). Otherwise, the conv_dilation_H = conv_dilation_W = conv_dilation. Default: conv_dilation = 1.

• conv_groups (int) – The groups number of the conv2d Layer. According to grouped convolution in Alex Krizhevsky’s Deep CNN paper: when group=2, the first half of the filters is only connected to the first half of the input channels, while the second half of the filters is only connected to the second half of the input channels. Default: groups=1.

• param_attr (ParamAttr|None) – The parameter attribute for learnable parameters/weights of conv2d. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as param_attr. If the Initializer of the param_attr is not set, the parameter is initialized with $$Normal(0.0, std)$$, and the $$std$$ is $$(\frac{2.0 }{filter\_elem\_num})^{0.5}$$. Default: None.

• bias_attr (ParamAttr|bool|None) – The parameter attribute for the bias of conv2d. If it is set to False, no bias will be added to the output units. If it is set to None or one attribute of ParamAttr, conv2d will create ParamAttr as bias_attr. If the Initializer of the bias_attr is not set, the bias is initialized zero. Default: None.

• act (str) – Activation type for conv2d, if it is set to None, activation is not appended. Default: None.

• use_cudnn (bool) – Use cudnn kernel or not, it is valid only when the cudnn library is installed. Default: True

Returns

The result of input after Convolution2d and Pool2d.

Return type

Variable

Examples

import paddle.fluid as fluid
img = fluid.layers.data(name='img', shape=[1, 28, 28], dtype='float32')
conv_pool = fluid.nets.simple_img_conv_pool(input=img,
filter_size=5,
num_filters=20,
pool_size=2,
pool_stride=2,
act="relu")

format: plain


text:
# Disorder-induced suppression of superconductivity in theSi(111)−(7×3)-In surface: Scanning tunneling microscopy study

@article{Yoshizawa2015DisorderinducedSO,
title={Disorder-induced suppression of superconductivity in theSi(111)−(7×3)-In surface: Scanning tunneling microscopy study},
author={Shunsuke Yoshizawa and Howon Kim and Yukio Hasegawa and Takashi Uchihashi},
journal={Physical Review B},
year={2015},
volume={92},
pages={041410}
}
The critical effect of disorder on the two-dimensional (2D) surface superconductor Si(111)-($\sqrt{7}\times\sqrt{3}$)-In is clarified by comparing two regions with different degrees of disorder. Low-temperature scanning tunneling microscopy measurements reveal that superconductivity is retained in the less disordered region, judging from the characteristic differential conductance ($dI/dV$) spectra and from the formation of vortices under magnetic fields. In striking contrast, the absence of… Expand
10 Citations

#### Figures and Tables from this paper

Influence of disorder on superconductivity in the Si(111)- 7× 3-In surface
To uncover the critical effect of disorder on superconductivity, the ideal method is to visualize the microscopic crystalline deficiencies in real space while measuring the macroscopicExpand
Superconductivity of metal-induced surface reconstructions on silicon
Recent progress in superconducting metal-induced surface reconstructions on silicon is reviewed, mainly focusing on the results of the author's group. After a brief introduction of anExpand
Electronic and transport properties of Pb-dense reconstructions on Si(100)
Abstract Electronic and transport properties of the dense Pb/Si(100) reconstructions, 2 × 1-Pb with 1.5 ML (monolayer) of Pb and c(4 × 4)-Pb with ∼ 1.875 (15/8) ML of Pb, have been explored usingExpand
Phonon limited superconducting correlations in metallic nanograins
• Materials Science, Medicine
• Scientific reports
• 2015
It is demonstrated that strong e-ph scattering smears the peak structure in the electronic density-of-states of a metallic grain and enhances the electron mass, and thereby limits the highest Tc achievable by quantum confinement. Expand
Superconducting single-atomic-layer Tl-Pb compounds on Ge(111) and Si(111) surfaces
• T. Nakamura, +4 authors S. Hasegawa
• Materials Science
• Applied Surface Science
• 2019
Abstract The set of known single-atomic-layer (SAL) superconductors has been limited so far by a few examples, including Si(111)-hex- 7 × 3 -In reconstruction, Si(111)-SIC-Pb and -HIC-Pb phases, andExpand
Superconducting single-atomic-layer Tl-Pb compounds on Ge(111) and Si(111) surfaces
The set of known single-atomic-layer (SAL) superconductors has been limited so far by a few examples, including Si(111)-hex× 7 3 -In reconstruction, Si(111)-SIC-Pb and -HIC-Pb phases, and Si(111) × 3Expand
Atomically thin half-van der Waals metals enabled by confinement heteroepitaxy
Large-area, environmentally stable, single-crystal 2D gallium, indium and tin that are stabilized at the interface of epitaxial graphene and silicon carbide that demonstrate compelling superconducting properties are demonstrated. Expand


text:
### Home > CALC > Chapter 4 > Lesson 4.4.1 > Problem4-130

4-130.

Evaluate.

1. $\lim\limits_ { x \rightarrow 9 } \frac { \sqrt { x } - 3 } { x - 9 }$

You could multiply the numerator and denominator by the conjugate of the numerator.

Or... you could recognize that this is Ana's Definition of the Derivative.

1. $\lim\limits_ { h \rightarrow 0 } \frac { \sqrt { 2 + h } - \sqrt { 2 } } { h }$

This is Hana's Definition of the Derivative.
What is $f(x)$?
What is $a$?
What is $f^\prime(x)$?
The limit $= f^\prime(a) =$_________

1. $\lim\limits_ { x \rightarrow \infty } \frac { 2 \sqrt { x } + 1 } { 5 - \sqrt { x } }$

This limit is approaching infinity. What is the end behavior? Is there a horizontal asymptote? Compare the highest-power terms in the numerator and denominator. Pay attention to coefficients.

$=-2$

1. $\lim\limits_ { x \rightarrow \infty } \operatorname { cos } x$

Visualize the graph of $y =\operatorname{cos}x$. It oscillates as $x→∞$.


text:
It is currently 23 Sep 2017, 09:53

# Happening Now:

Alleviate MBA app anxiety! Come to Chat Room #2

### GMAT Club Daily Prep

#### Thank you for using the timer - this advanced tool can estimate your performance and suggest more practice questions. We have subscribed you to Daily Prep Questions via email.

Customized
for You

we will pick new questions that match your level based on your Timer History

Track

every week, we’ll send you an estimated GMAT score based on your performance

Practice
Pays

we will pick new questions that match your level based on your Timer History

# Events & Promotions

###### Events & Promotions in June
Open Detailed Calendar

# If |y-1/2| < 11/2, which of the following could be a value

Author Message
TAGS:

### Hide Tags

e-GMAT Representative
Joined: 04 Jan 2015
Posts: 746

Kudos [?]: 2010 [1], given: 123

Re: If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

08 May 2015, 05:56
1
KUDOS
Expert's post
Here's a visual solution for this question:

$$|y-\frac{1}{2}|$$ represents the distance of y from the point $$\frac{1}{2}$$ = 0.5 on the number line.

The given inequality tells us that the distance of y from the point 0.5 is less than $$\frac{11}{2}$$ (that is, 5.5)

The points that are at a distance of 5.5 from 0.5 on the number line are: -5 and 6

This means, -5 < y < 6

Therefore, Option C is the correct answer.
_________________

| '4 out of Top 5' Instructors on gmatclub | 70 point improvement guarantee | www.e-gmat.com

Kudos [?]: 2010 [1], given: 123

Senior Manager
Status: Professional GMAT Tutor
Affiliations: AB, cum laude, Harvard University (Class of '02)
Joined: 10 Jul 2015
Posts: 393

Kudos [?]: 459 [0], given: 53

Location: United States (CA)
Age: 37
GMAT 1: 770 Q47 V48
GMAT 2: 730 Q44 V47
GMAT 3: 750 Q50 V42
GRE 1: 337 Q168 V169
WE: Education (Education)
Re: If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

11 Apr 2016, 16:35
If you are confused by the algebraic solution, then it might be easier to plug in when absolute values are involved.
Attachments

Screen Shot 2016-04-11 at 4.33.56 PM.png [ 107.07 KiB | Viewed 530 times ]

_________________

Harvard grad and 770 GMAT scorer, offering high-quality private GMAT tutoring, both in-person and via Skype, since 2002.

McElroy Tutoring

Kudos [?]: 459 [0], given: 53

Target Test Prep Representative
Status: Founder & CEO
Affiliations: Target Test Prep
Joined: 14 Oct 2015
Posts: 1537

Kudos [?]: 778 [2], given: 2

Location: United States (CA)
Re: If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

03 May 2016, 07:55
2
KUDOS
Expert's post
Bunuel wrote:
If $$|y-\frac{1}{2}| < \frac{11}{2}$$, which of the following could be a value of y?

(A) -11
(B) -11/2
(C) 11/2
(D) 11
(E) 22

Solution:

We must solve the inequality for two cases:

Case 1: (y-1/2) is Positive:

|y – ½| < 11/2

y – ½ < 11/2

Add ½ to both sides of the equation:

y < 6

Case 2: (y-1/2) is Negative:

|y – ½| < 11/2

-(y – ½) < 11/2

-y + ½ < 11/2

Subtract ½ from both sides of the equation:

-y < 5

y > -5

So we know that y is greater than -5 and less than 6. The only answer choice that fits this criterion is answer choice C, 11/2.

_________________

Scott Woodbury-Stewart
Founder and CEO

GMAT Quant Self-Study Course
500+ lessons 3000+ practice problems 800+ HD solutions

Kudos [?]: 778 [2], given: 2

GMAT Club Legend
Joined: 09 Sep 2013
Posts: 17620

Kudos [?]: 271 [0], given: 0

Re: If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

09 May 2017, 10:02
Hello from the GMAT Club BumpBot!

Thanks to another GMAT Club member, I have just discovered this valuable topic, yet it had no discussion for over a year. I am now bumping it up - doing my job. I think you may find it valuable (esp those replies with Kudos).

Want to see all other topics I dig out? Follow me (click follow button on profile). You will receive a summary of all topics I bump in your profile area as well as via email.
_________________

Kudos [?]: 271 [0], given: 0

Senior Manager
Joined: 04 Sep 2016
Posts: 270

Kudos [?]: 42 [0], given: 164

Location: India
WE: Engineering (Other)
Re: If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

30 Aug 2017, 07:48
Bunuel shashankism Engr2012

Hi Experts, how much I try I can not refrain from opening the modulus and
assigning +ve and -ve values to expressions inside modulus. Is there any
short cut approach to this , please see screenshot.
Attachments

20170830_201214.jpg [ 2.34 MiB | Viewed 71 times ]

_________________

Press kudos if you liked this post

Kudos [?]: 42 [0], given: 164

Senior Manager
Joined: 04 Sep 2016
Posts: 270

Kudos [?]: 42 [0], given: 164

Location: India
WE: Engineering (Other)
If |y-1/2| < 11/2, which of the following could be a value [#permalink]

### Show Tags

31 Aug 2017, 09:05
hi EgmatQuantExpert

I am able to understand |x| < a means -a < x < a
but falter when taking complex inequalities like |x-a| < b

WR,
Arpit
_________________

Press kudos if you liked this post

Kudos [?]: 42 [0], given: 164

If |y-1/2| < 11/2, which of the following could be a value   [#permalink] 31 Aug 2017, 09:05

Go to page   Previous    1   2   [ 26 posts ]

Similar topics Replies Last post
Similar
Topics:
1 If = 3, which of the following could be the value of x – 4? 9 08 Oct 2016, 14:26
2 If 4x>y and y<0 then which of the following could be the value of x/4 5 05 Apr 2017, 13:36
1 If a^2 + 7a < 0, then which one of the following could be the value of 2 10 Mar 2017, 10:33
11 If a^2b > 1 and b < 2, which of the following could be the value of a? 11 22 Jul 2017, 06:18
2 If x^2 < x, which of the following could be a value of x? 10 08 Aug 2017, 20:45
Display posts from previous: Sort by


text:
# What is the normalized graph matrix if the row-sum of proximity matrix is zero?

Let $$S \in \mathbb{R}_{\ge 0}^{n \times n}$$ be the proximity (or similarity) matrix of a graph, e.g. $$S = \left[ \begin{matrix} 0 & 0.9 & 0.3 \\ 0.9 & 0 & 0.4 \\ 0.3 & 0.4 & 0 \end{matrix}\right]$$ Here, the elements on the diag should be $$0$$s.

Now, we define $$D$$ be a diagonal matrix with the row-sum of $$S$$ on the diagonal entries. In my running example, $$D = \left[ \begin{matrix} 1.2 & 0 & 0 \\ 0 & 1.3 & 0 \\ 0 & 0 & 0.7 \end{matrix}\right]$$

Then, we normalize $$S$$ with $$D$$, i.e. \begin{align} \bar{S} &= D^{-\frac{1}{2}} S D^{-\frac{1}{2}} \\ &= \left[ \begin{matrix} \frac{1}{\sqrt{1.2}} & 0 & 0 \\ 0 & \frac{1}{\sqrt{1.3}} & 0 \\ 0 & 0 & \frac{1}{\sqrt{0.7}} \end{matrix}\right] \left[ \begin{matrix} 0 & 0.9 & 0.3 \\ 0.9 & 0 & 0.4 \\ 0.3 & 0.4 & 0 \end{matrix}\right] \left[ \begin{matrix} \frac{1}{\sqrt{1.2}} & 0 & 0 \\ 0 & \frac{1}{\sqrt{1.3}} & 0 \\ 0 & 0 & \frac{1}{\sqrt{0.7}} \end{matrix}\right] \\ \end{align} (omit subsequent steps...)

My question is: How can I handle when the row-sum of $$S$$ is zero?

Specifically, if $$S = \left[ \begin{matrix} 0 & 0 & 0.3 \\ 0 & 0 & 0 \\ 0.3 & 0 & 0 \end{matrix}\right]$$ i.e. no edge is connected to vertex 2. Thus, $$D$$ will be $$D = \left[ \begin{matrix} 0.3 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0.3 \end{matrix}\right]$$ As a result, I don't know how to compute $$D^{-\frac{1}{2}}$$. 😔 So, what is the correct calculation method for $$D^{-\frac{1}{2}}$$ and $$\bar{S}$$ hereafter at this time?

• Not my field, but are you referring to the Graph Distance Matrix? Would a zero column indicate that a given vertex was distance zero from every other vertex? Is that permitted in such a matrix? – Theo Bendit Apr 23 at 11:51
• @TheoBendit Maybe Graph Distance Matrix is what I refer. I look up wikipedia and find a sentence in the 2nd para., it says "For the isolated vertices (those with degree 0), a common choice is to set the corresponding element ${\textstyle L_{i,i}^{\text{rw}}}$ to 0." Is it right for my question? – Lizhi Liu Apr 23 at 12:16
• It doesn't look right for your question. The Laplacian matrix has negative entries, whereas your matrices don't seem to. Could you define what a proximity matrix is? Am I correct in thinking that it's an $n \times n$ matrix $A$ defined by $n$ points $x_1, \ldots, x_n$ where $A_{ij} = d(x_i, x_j)$? – Theo Bendit Apr 23 at 12:26
• @TheoBendit Yes, your understanding is right! – Lizhi Liu Apr 23 at 12:30
• OK, so then the $i$th column totalling zero can only happen if every entry is $0$, since every distance is non-negative. This means that $d(x_i, x_j) = 0$ for all $j$, which presumably means that $x_i = x_j$ for all $j$. That is, every point is equal to every other point, and the whole matrix is the zero matrix. This seems like a pretty degenerate case, and I suspect that such a case is either purposefully excluded, or already normalised without having to conjugate by $D^{\frac{1}{2}}$. – Theo Bendit Apr 23 at 12:34

(This is just an expansion on the points in the comments above, so that the question has a proper answer.)

Given $$n$$ points $$x_1, \ldots, x_n$$ in a metric space $$(X, d)$$ (e.g. $$X = \Bbb{R}^m$$ with the Euclidean distance function), the proximity matrix $$A$$ is defined by $$A_{ij} = d(x_i, x_j)$$. Naturally, the symmetry of metrics imply that $$A$$ must be symmetric, and the positive-definiteness of the metric implies that the entries of $$A$$ are non-negative, with $$0$$s down the diagonal.

However, not every symmetric matrix with non-negative entries and $$0$$s down the diagonal will be a proximity matrix. Indeed, the example provided: $$S = \begin{pmatrix} 0 & 0 & 0.3 \\ 0 & 0 & 0 \\ 0.3 & 0 & 0 \end{pmatrix}$$ cannot be a proximity matrix, as we have $$d(x_1, x_3) = 0.3 > 0 + 0 = d(x_1, x_2) + d(x_2, x_3),$$ violating triangle inequality.

Indeed, if the $$j$$th column of the matrix sums to $$0$$, then we have $$d(x_i, x_j) = 0$$ for all $$i$$, as each entry is non-negative. By the definiteness of the metric, this implies $$x_i = x_j$$ for all $$i$$. That is, all the points are the same! In such cases, every entry in the proximity matrix must be $$0$$.

Can this ever happen? Sure, if all the points are the same. If not, then all the column sums are strictly positive, and computation of $$D^{-\frac{1}{2}}$$ is straight forward.

Is this allowed to happen? Maybe; it depends on the circumstances. Certainly such a case is pretty degenerate, and I could foresee many applications of proximity matrices rejecting such a case, or treating it specially.

Ultimately, it's up to the individual to decide what their proximity matrix is modelling, and what every point being identical says about the situation they're modelling.

In this case, you indicated that this was modelling protein interactions, with $$0$$ columns appearing when data was missing. It's difficult to tell without a much wider perspective, but it would appear that filling in missing data with zeros breaks the proximity matrix, for the reasons argued above (it makes the matrix not a valid proximity matrix). If you don't have data on a given protein, then try performing your analysis with one fewer protein.

• Thank you for ur very long supplement! Grateful! – Lizhi Liu Apr 24 at 5:17


text:
# A cube made of an unknown material has a height of 9 cm. The mass of this cube is 3,645 grams. How do you calculate the density of this cube given this information?

$\text{Density, "rho="Mass"/"Volume}$
So given that it is a cube, $\rho = \frac{3645 \cdot g}{{\left(9 \cdot c m\right)}^{3}}$ $=$ $5 \cdot g \cdot c {m}^{-} 3$.


text:
## Q. 6.4

Ductility of an Aluminum Alloy
The aluminum alloy in Example 6-1 has a final length after failure of 2.195 in. and a final diameter of 0.398 in. at the fracture surface. Calculate the ductility of this alloy.

## Verified Solution

% Elongation = $\frac{l_{f} \ – \ l_{0}}{l_{0}} × 100=\frac{2.195 \ – \ 2.000}{2.000} × 100= 9.75$%
% Reduction in area= $\frac{A_{0} \ – \ A{f}}{A_{0}} × 100$
$=\frac{(\pi /4)(0.505)^2 \ – \ (\pi /4)(0.398)^2}{(\pi /4)(0.505)^2} × 100$
$= 37.9$%

The final length is less than 2.205 in. (see Table 6-1) because, after fracture, the elastic strain is recovered.

 Table 6-1 The results of a tensile test of a 0.505-in. diameter aluminum alloy test bar, initial length ($l_{0}$) = 2 in. Load (lb) Change in Length (in.) Calculated Stress (psi) Strain (in./in.) 0 0.000 0 0 1000 0.001 4,993 0.0005 3000 0.003 14,978 0.0015 5000 0.005 24,963 0.0025 7000 0.007 34,948 0.0035 7500 0.030 37,445 0.0150 7900 0.080 39,442 0.0400 8000 (maximum load) 0.120 39,941 0.0600 7950 0.160 39,691 0.0800 7600 (fracture) 0.205 37,944 0.1025


text:
# Fatou coordinate

(Redirected from Fatou coordinates)

Fatou coordinate - a term predominantly used in holomorphic dynamics - usually refers to one of the $2m$ principal Abel functions of a holomorphic function $f$ with parabolic fixpoint (which we assume for simplicity to be located at 0): $$f(z)=z+c_{m+1}z^{m+1} + c_{m+2}z^{m+2} + \dots$$

## Literature

• Milnor, J. (2006). Dynamics in one complex variable. 3rd ed. Princeton Annals in Mathematics 160. Princeton, NJ: Princeton University Press. viii, 304 p.


text:
# Tag Info

32

Side complaint: I seem to be getting an awful lot of review audits, maybe one per five legitimate review requests. If you are asking me to spend my time performing community service, I would prefer you not waste my time with pointless "fake" tasks.

29

The newest and one of the most ridiculous examples to date: https://math.stackexchange.com/review/first-posts/109542 I failed this by clicking "add comment". That's right. You're not allowed to comment when you're being audited. I'm filing a bug report.

25

I have failed a few. One interesting thing is that, for "close" audits at least, the system lies about the vote count, showing 0 instead of the actual vote count. Which seems to be counterproductive: it appears they change the vote count to encourage reviewers to evaluate the question independently, but then they criticize the reviewer being too independent ...

20

https://math.stackexchange.com/a/508096/17596 showed up as a "first post" review. I found the animation far more distracting than helpful, so I downvoted (disagreeing, apparently, with fifteen folks who think it's the cat's meow). Review audit failed.

18

I failed a review audit for this question because I went to close it as duplicate as I recognised the title and content of the post so thought it was an exact duplicate of itself (the review had artificially made it appear that the question was written 2 hours ago). This highlights a flaw in the system in my opinion.

16

Bad robo audits can be frustrating, but they don't really do much, apart from make you wish the robo auditor had a neck you could squeeze. The criteria for how it chooses the "right" choice is explained here and is clearly prone to occasional errors. (If it weren't, why would we need reviewers?) It is possible to get an automatic review ban for failing ...

16

Just to get this question off the unasnwered list, here's the explanation (thanks to YuiTo Cheng, Xander Henderson, jgon and other commenters for the help). It turns out that the poster of the answers had been posting a large number of word-for-word identical answers on similar reference-request questions. The link contains a referral, making it seem quite ...

15

I'd say the culprit is that last sentence: This much anticipated second edition of the most successful calculus text published in the last two decades retains the best of the first edition while introducing important advances and refinements. That just reeks of advertisement. It says absolutely nothing of content about the book itself, and sounds like it ...

13

No, you did right. That's not an answer in any way, and voting to delete it is the correct action. The poor audit system was misled by the post having a score of +5. Since it doesn't know any mathematics, the only criterion it has to pick posts for audits is the vote tally. Sometimes that leads to wrong-headed audits. But that's at least as much an issue ...

12

The first time I got an audit, I thought it was a good idea. Well, not anymore. I've failed some when we changed the policy, initiated by - I believe - this topic. Since the new "close" reason was included, I go for it almost on every question that was nominated for closing and lacks any context or effort (i.e., copy/paste questions). I myself nominate such ...

8

I just "failed" a reopen review audit. My philosophy is that the principle of res judicata applies to all reopen requests: that is, I will automatically reject any request to reopen a question that has not been edited since it was closed, unless strong reasons to reopen are given in the comments. The audit I failed showed no edits since closure and no ...

8

This question post is a horrible "exemplar" of a post that should be reopened, according to a "reopen review queue audit." In fact, the question should be put on hold, so the asker can improve it, and perhaps, and only then, reopened. Including it as an exemplar of a post that users should reopen teaches users to reopen very poor posts, which is ...

8

Because sometimes there is a bigger pattern that regular users don't see where certain websites, books, or otherwise commercial venues post their links here. What might seem to you as a single post with a valid link can easily be considered spam when zooming out. Unfortunately, regular users don't have these tools to zoom out (except long-term users with ...

7

I rarely review, but I thought I'd give it a try, and almost immediately failed an audit. I was presented with a very poor answer in the “late answers” queue. I opened the thread to which it was attached. I saw that the answer I was reviewing had been deleted several weeks ago, so I clicked the “no action necessary” button, and got this obnoxious response: ...

7

I failed a First Post with this question for one of two reasons. I noticed two things at first glance: There was a mix of unTeXified and TeXified variables. The post mentioned that the numbers could be "very large," but didn't specify how large they could be, nor give much other context aside from "it's for a computer program." What I intended to do (in ...

7

Probability and expectancy problem This is supposed to be a "reopen queue audit question" and to "pass" the audit, one is supposed to vote to reopen. This is a problem statement question and deserves to be downvoted and closed, and not reopened!

6

The question is "missing context," as such it can be voted to be closed within the policies of the site (even if not everybody agrees with this policy). However, sometimes question that are interesting or unusual can be well received even without context. Abstractly, not few are of the opinion that unusually positive aspects in a question can compensate ...

6

Your edit nails it. The post had a spam flag on it that was handled as helpful, and thus the audit system thought that the post was spam (deleted post with a valid spam flag = BAD!). Although the spam flag on the original version was not without merit, I have cleared the spam flag from the post (it caused three audit failures altogether). That had the side ...

5

I failed this one. After the fact, I voted to close it and it has since been closed. https://math.stackexchange.com/review/close/246038 It has no context or details. I hope the fact that it is now closed removes it from the review audit.

5

https://math.stackexchange.com/review/close/1076507 The entire question reads: I have been stuck on this problem for hours and have absolutely no clue how to go about it. Any help would be amazing. The question itself is in the title, and is a pretty standard combinatorics question. I honestly don't understand how the question garnered five upvotes in ...

5

https://math.stackexchange.com/review/close/1056774 The entire question reads: Are there any necessary or sufficient condition for when in a general convex hexagon, the lines formed by joining the midpoints of opposite sides are concurrent (intersect in a common point)? This question lacks context. I can see that it might be interesting, but it appears ...

4

I failed this audit because I wanted it add a comment. Not that I added a bad comment. For all any computer might know I wanted to say this problem is the best research direction for mathematics since the Langland's program. Que sera sera.

4

I failed this one, ironically because I put too much effort into it. It was presented as a first post with 0 votes, on a question that had been answered 10 days ago. I navigated to the question and saw it had good answers v similar to the one I was reviewing so tried to add a comment to say it had already been answered => FAIL. (What I should have spotted ...

4

https://math.stackexchange.com/review/close/1015074 This question is a fairly straight-forward "problem statement question" that I voted to close as "Lacking Context." I think that the level of the material (advanced undergraduate or introductory graduate level complex analysis) coupled with the (non-obvious) difficulty of the problem earned it several ...

4

Yesterday I failed this Review audit, and when I posted an answer in the Reopen/Undelete thread, another user reported failing two similar audits, and someone else later failed another one. That same person failed another one of the same type less than 6 hours later. The number of such posts by the same user (who also had a question deleted on MO, though I ...

4

Are there any properties of sup? The question title is not informative, and suggests a very broad question (rather than the fairly focused question which is actually presented). The question itself reads: Can I say: $$\sup |f(x)-h(x)+g(x)-g(x)| = \sup|f(x)-g(x)|+ \sup|g(x)-h(x)|$$ I can't seem to find any properties of $\sup$ , so i am wondering ...

3

I failed this one: https://math.stackexchange.com/review/low-quality-posts/512046 I understand that it is an interesting question that does provide a bit of background with a link to the problem. But there is no attempt at solving the problem, so I voted to close. Some will disagree on this, but the question doesn't (IMO) belong in the list of audit ...

3

https://math.stackexchange.com/review/first-posts/174827 I failed this audit, because I tried to mark it as a duplicate. The audit said it has been asked (I think) 1 hour ago and I was absolutely sure, that I have read exactly the same question a few days ago. However, I had read exactly this question before, because that first post was a fake of the ...

Only top voted, non community-wiki answers of a minimum length are eligible


text:
## LATEX-L@LISTSERV.UNI-HEIDELBERG.DE

#### View:

 Message: [ First | Previous | Next | Last ] By Topic: [ First | Previous | Next | Last ] By Author: [ First | Previous | Next | Last ] Font: Proportional Font

Subject:

Re: Shortref mechanism

From:

Date:

Sun, 2 Mar 1997 13:48:57 +0100

Content-Type:

text/plain

Parts/Attachments:

 text/plain (36 lines)
 Frank Mittelbach <[log in to unmask]> wrote: >The above means that the shortref mechanism either has to be fully >expandable or that one can't use it to produce glyphs that might play >a part in ligature or kerning tables.   I cannot follow the details in your reasoning, but I can note that with deterministic parsing, the method generally used in LaTeX, conditional parsing have such limits.   But with non-deterministic parsing more general things can be done:   For example, I just made a definition command that can produce commands having optional arguments; in this general approach, I had to switch from LaTeX style deterministic parsing to non-deterministic parsing.   A command then picks down a look-ahead portion, like to the next space     \def\foo#1 { } %% Putting back the space consumed by the argument. It then examines this argument, and then has the opportunity of rejecting whatever non-parsed stuff unaffected, by returning (a part of) the _original_ look-ahead portion. Certainly, spaces do count in this approach. If the the only problem is the next character (like a brace) after the command or active character, perhaps you can combine, doing a simple one-token look-ahead, before the argument pickup; otherwise, TeX requires arguments to not have an extra "}", so one cannot enclose such a command in braces without at least one space occurs within those braces. (So there must be a look-ahead token that you can ensure to be on the same group level as the command.)   Actually, what is needed, is a simple extension of the TeX \def command:     \def\name{definition text}[exception text] If a command somehow is unable to process the parameter text, then the original input text (before the parsing of the parameter text began) is left unaffected, and the execution passes to the exception text.   Then stuff like optional commands could be implemented most easily, and perhaps the stuff you are asking for, too.   Hans Aberg


text:
# Missing data imputation in time series in R

I have got hourly temperature data from 2012 to 2016 as follows:

> head(htemp)
HH_ID      TEMPERATURE   YY_ID    DD_ID       MM_ID
1 201201010000     8.98       2012    20120101     201201
2 201201010100     8.67       2012    20120101     201201
3 201201010200     8.69       2012    20120101     201201
4 201201010300     8.50       2012    20120101     201201
5 201201010400     8.30       2012    20120101     201201
6 201201010500     8.10       2012    20120101     201201

There are missing data in the data

  1. Missing data(NA) on individual hour (on 201209281400, 201209290000...)
2. Missing data on consecutive hours(like no observation(NA) on 201210241700, 201210241800, 201210241900).
3. No observation(NA) on a whole day(20130328).

I am wondering how to interpolate the missing data using adjacent data, i.e. linearly interpolation for individual missing HH data, and adopting the "typical" pattern from adjacent days for the whole day missing data (linearly interpolating each HH of the missing day using the temperature of corresponding HH in adjacent days).

## Edit

I would like to perform the time series analysis on the temperature data, like decomposing (stl), modelling (auto.arima) and forecasting (forecast) it as well. It seems stl cannot handle missing data, so I think it might be necessary to impute the missing data first.

I have another data set containing electricity demand, where there is no missing data. I may also model the demand data using temperature data as covariate.

## Update

I tried imp<-mice(htemp) on my data, but got an error:

iter imp variable
1   1  TEMPERATURE
Error in solve.default(xtx + diag(pen)) :
system is computationally singular: reciprocal condition number = 5.03072e-28

• I'd recommend using multiple imputation. Check out the MICE package. – StatsStudent Jul 16 '16 at 21:31
• Hi @StatsStudent, I simply tried mice(), which does not work here. Would you mind giving some more hints? Thanks. – Jeannie Jul 17 '16 at 11:19
• The error comes from the interrelated colums, e.g. MM directly follows from DD. For model based imputation, you would need to prepare the columns somewhat. – Michael M Jul 17 '16 at 11:43
• You might also want to include the purpose of your overall analysis. Perhaps imputation is not the correct answer. – Wayne Jul 17 '16 at 13:14
• @Wayne Thanks for your suggestion. I would like to perform time series on the temperature, and model another variable using temperature as covariate. It seems to me that imputing missing data at the very beginning will make the further analysis more convenient. – Jeannie Jul 17 '16 at 13:39

library(imputeTS)
x <- ts(htemp$TEMPERATURE, frequency = 12) x.withoutNA <- na.kalman(x)  This would be one possible solution of getting imputed temperature values. Here another one with the forecast package: library(forecast) x <- ts(htemp$TEMPERATURE, frequency = 12)


text:
# Lesson # 12. Recursive algorithms

Дата изменения: 15 августа 2022

Содержание:

## Theory

Sample 1: Executing Recursive Factorial by Hand (without recursion)

int factorial(int n) { if (n <= 0) { return 1; } return n * factorial(n - 1); }   int main() { int x = factorial(3); std::cout << x; }

Sample 2: Recursive Factorial

int factorial(int n) { // determine if n is less than or equal to 0 or not if (n <= 0) { // if so, then just answer 1 return 1; } //otherwise else { // compute (n-1)! , call it n_minus1_fact int n_minus1_fact = factorial(n - 1); // multiply n_minus1_fact \8n // the resulting product is the answer return n_minus1_fact* n; } }   int main() { int x = factorial(3); std::cout << x; }

Lab 1: Simple recursion
To do: You should calculate the numbers between x and y, which are entered by user. You are given a code how to do it without recursion, just using loop. You should transfer this code to complete the task, using recursion.

Expected output:

Enter x and y, x<y
2 4
Sum = 9

✍ How to do:

1. Create an empty console application.
2. Type the following code to complete the task without recursion and test it:
3. int main()
{
int x,y;
std::cout << "Enter x and y, x<y";
std::cin >> x;
std::cin >> y;

int sum = 0;
for (int i = x; i <= y; i++) {
sum += i;
}
std::cout << "Sum = " << sum;
}

4. After, comment this code out, using /* ... */. We don't need this code anymore.
5. Create a function called recursive_sum with two parameters, they are x and y.
6. int recursive_sum(int x, int y) { }
7. In order not to make an infinite repetition of the recursive function we need to place the condition there, it is called base case:
8. if (x == y) { return x; }
9. The base case will break our recursion.
10. Then, you should add the main part, the recursion. So, you should invoke the function itself:
11. return x + recursive_sum(x + 1, y);
12. Return to the main function and place the code to call the function after x and y initiliazing code:
13. std::cout << "Sum = " << recursive_sum(x, y);
14. Run the application and check the output.
Lab 2: Recursive Fibonacci sequence

To do: You should output the n-th number of Fibonacci sequence (n is a positive integer). You should complete the task, using recursion.

Note: Fibonacci sequence:

0 1 2 3 5 8 13 21...
Each specified number is the sum of two preceding numbers of the sequence.

Expected output:

enter a number:
6
6! = 8

✍ How to do:

1. Create an empty console application.
2. Create a function called fib with one parameter, which is n - the order of a sequence number ti output.
3. int fib(int n) { }
4. In order not to make an infinite repetition of the recursive function we need to place the condition there, it is called base case. We have two of them: when n = 0 and when n = 1:
5. if (n == 1) { return 1; } else if (n == 0) { return 0; }
6. The base case will break our recursion.
7. After, we should create a code to output the result in the case when n > 1. In that case we should compute the sum of two preceding numbers of the sequence. In recursive meaning it means that we should invoke the same function for n-1 and for n-2 and add the results:
8.  else if (n > 1) { return fib(n - 1) + fib(n - 2);   }
9. Return to the main function and place the code to input n and call the function:
10. std::cout << "enter a number: \n"; int n; std::cin >> n; std::cout << n << "! = " << fib(n);
11. Run the application and check the output.
Вставить формулу как
Дополнительные настройки
Цвет формулы
Используйте LaTeX для набора формулы
Предпросмотр
$${}$$
Формула не набрана
Вставить


text:
## Stream: maths

### Topic: Subring inclusion

#### Adam Topaz (Aug 31 2020 at 21:33):

Suppose I have a ring R and a (bundled) subring S : subring R. What's the name for the canonical inclusion S ->+* R? I'm sure this is in mathlib, but I can't seem to find it in ring_theory.subring

#### Adam Topaz (Aug 31 2020 at 21:36):

Got it. It's called subring.subtype for some reason...

#### Johan Commelin (Sep 01 2020 at 03:28):

@Adam Topaz Yup, those foo.subtype names are really confusing. I think we should rename all of them. But never got around to doing it.

Last updated: May 11 2021 at 15:12 UTC


text:
## Top new questions this week:

### Why do volatility and correlation increase in times of crisis?

can somebody please explain to me why volatility and correlation increase in times of crisis? It is connected somehow to the herding effect. But I cannot really explain it. And also why are negative ...

volatility correlation

### portfolio optimization averaging weights

I'm playing around with different portfolio optimization techniques. Amongst others I was also looking at the resampling method, especially the one described in Meucci. I have two general questions ...

modern-portfolio-theory portfolio-optimization

### How to price a path dependent exchange option using?

Assume you have two stocks $S$ and $P$ so that at initial time $t = 0$: $S_0 > P_0$. You bought an option which pays off $S_T - P_T$ as long as $S_t > P_t$ through the time $0 < t < T$. ...

options option-pricing no-arbitrage-theory

### Calibrating stochastic volatility model from price history (not option prices)

For stochastic volatility models like Heston, it seems like the standard approach is to calibrate the models from option prices. This seems a bit like a chicken and an egg problem -- wouldn't we ...

volatility stochastic-volatility

### Clarify a derivation in Pat Hagan's Convexity Conundrums

I am looking for help in understanding the algebraic derivation to go in between some of the lines in Pat Hagan's famous Convexity Conundrums paper e.g. how he goes from 3.4a to 3.5a.

mathematics

### Equivalent Definitions of Self-Financing Portfolio

Consider a multi-period model with $t=0,...,T$. Suppose there is a bond with $B_0=1$ and $B_t=(1+R)^t$ and a stock with $S_0=s_0$ and $$S_{t+1}=S_t\,\xi_{t+1},$$ with $\xi_t$ iid random ...

portfolio no-arbitrage-theory self-study

### How does Algorithmic Differentiation work and where can it be applied?

The title says it all, but let me expand on it. Algorithmic differentiation seems to be a method that allows a program / compiler to determine what the derivative is of a function. I imagine it's a ...

greeks algorithmic-derivative

## Greatest hits from previous weeks:

### What is a stationary process?

How do you explain what a stationary process is? In the first place, what is meant by process, and then what does the process have to be like so it can be called stationary?

stochastic-calculus time-series stationarity

### Switching from C++ to R - limitations/applications

I've only recently begun exploring and learning R (especially since Dirk recommended RStudio and a lot of people in here speak highly of R). I'm rather C(++) oriented, so it got me thinking - what are ...

r development

### Comparing volatility using GARCH

Hi, I am investigating if the policy of inflation targeting lowers the volatility of inflation. I have ran the GARCH regression: Inflation(t) = C + Inflation(t-1) + Inflation(t-2) + Error, but ...

volatility time-series garch econometrics inflation
 asked by Kunal Chauhan 1 vote

### Fair Price CDS Spread for a Bank

I have been using CreditGrades to calculate fair one year CDS spreads for firms. However, the authors of the model explicitly say that the model does not hold for banks or financial firms. If I need ...

derivatives cds default
 asked by varun chandra 1 vote

### Validating a Credit Scoring Model without Data

Fellow Quants, Suppose you have a credit scoring model that is developed without the aid of statistics, because (unfortunately) there is no historical default/loss data in your portfolio. The ...

models credit credit-scoring validation


text:
# Lecture 23 — Recursion¶

## Overview¶

• When a function calls itself, it is known as a recursive function.
• Use of the function call stack allows Python to handle recursive functions correctly.
• Examples include factorial, Fibonacci, greatest common divisor, flattening a list of lists, and mergesort.
• We’ll think about how to hand-simulate a recursive function as well as rules for writing recursive functions.

## Our First Example¶

• Consider the following Python function.

def blast(n):
if n > 0:
print(n)
blast(n-1)
else:
print("Blast off!")

• What is the the output from the call?

blast(5)

## Python’s Call Stack Mechanism¶

The following mechanism helps us understand what is happening:

• Each time the code makes a function call, Python puts information on the “call stack”, including
• All values of parameters and local variables
• The location in the code where the function call is being made.
• Python then makes the function call, switching execution to the start of the called function.
• This function in turn can make additional (potentially recursive) function calls, adding information to the top of the stack each time.
• When a function ends, Python looks at the top of the stack, and
• restores the values of the local variables and parameters of the most recent calling function,
• removes this information from the top of the stack,
• inserts the returned value of the called function (if any) in the appropriate location of the calling function’s code, and
• continues execution from the location where the call was made.

## Practice Problems to Illustrate This¶

What are the outputs of the following?

def rp1( L, i ):
if i < len(L):
print(L[i], end=' ')
rp1( L, i+1 )
else:
print()

def rp2( L, i ):
if i < len(L):
rp2( L, i+1 )
print(L[i], end=' ')
else:
print()

L = [ 2, 3, 5, 7, 11 ]
rp1(L,0)
rp2(L,0)

Note that the entirety of list L is not copied to the top of the stack. Instead, a reference (an alias) to L is placed on the stack.

## Factorial¶

• The factorial function is

$n! = n (n-1) (n-2) \cdots 1$

and

$0! = 1$
• This is an imprecise definition because the ... is not formally defined.

• Writing this recursively helps to clear it up:

$n! = n \cdot (n-1)!$

and

$0! = 1$

The factorial is now defined in terms of itself, but on a smaller number!

• Note how this definition now has a recursive part and a non-recursive part:

• The non-recursive part is called the base case. There must be at least one base case in every recursive function definition.

## Exploring Factorial¶

We will:

• Write a recursive Python function to implement $$n!$$.
• Hand-simulate the call stack for $$n=4$$.

We’ll add output code to the implementation to help visualize the recursive calls in a different way.

## Guidelines for Writing Recursive Functions¶

1. Define the problem you are trying to solve in terms of smaller / simpler instances of the problem. This includes
1. What needs to happen before making a recursive call?
2. What recursive call(s) must be made?
3. What needs to happen to combine or generate results after the recursive call (or calls) ends?
2. Define the base case or cases.
3. Make sure the code is proceeding toward the base case in every step.

## Fibonacci¶

• The Fibonacci sequence starts with the values 0 and 1.

• Each new value in the sequence is obtained by adding the two previous values, producing

$0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, \ldots$
• Recursively, the $$n^\text{th}$$ value, $$f_n$$, of the sequence is defined as

$f_n = f_{n-1} + f_{n-2}$
• This leads naturally to a recursive function...

## Dangers of Recursion¶

• Some recursive function implementations contain wasteful repeated computation.
• Recursive function calls — like any function calls — typically involve hidden overhead costs.
• Often, therefore, a recursive function can (and should) be replaced with a non-recursive, iterative function that is significantly more efficient.
• This is easy to do for both Factorial and Fibonacci, as we will see in class.

## Why, Then, Do We Study Recursion?¶

• Many of our definitions and even, our logical structures (such as lists), are formalized using recursion.
• Sometimes recursive functions are the first ones we come up with and the easiest to write (at least after you are comfortable with recursion).
• Only later do we write non-recursive versions.
• Sometimes on harder problems it is difficult to even write non-recursive functions! The list flattening problem below is one such example.

We’ll spend the rest of class on three more advanced examples:

1. Recursive geometric shapes: the Sierpinski triangle
2. Flattening a nested list
3. A recursive version of merge sort

## Recursive Geometric Shapes¶

• Fractals are often defined using recursion. How do we draw a Sierpinski triangle like the one shown below?

• We will look at this example in class and attempt to define the recursion.

• To aid us, we’ll look at a Tkinter Python program that implements the drawing of the Sierpinski triangle.

## Flattening a Nested List¶

• Suppose we want to take a list such as

v = [[1,5], 6, [[2]], [3, [7, 8, [9,10], [11,12] ]]]

and “flatten” it, converting it to just a list of values with no sublists.

v = [ 1, 5, 6, 2, 3, 7, 8, 9, 10, 11, 12 ]

• This is challenging because we don’t know when we write a function to solve this problem how “deep” the nesting of sublists goes. The solution should handle arbitrary depths:

• Many, many data structures (containers) in computer science have this type of nested / recursive structure: an entry in a list may be another list....
• To solve this problem we will also need to know how to recognize when an entry in a list is another list. For this we need to use the type function in Python. The following example should make this clear:

>>> v = [ 'a', 'b', 'c']
>>> x = 12
>>> type(v) == list
True
>>> type(v[0]) == list
False
>>> type(x) == int
True

• Now we are ready to solve the “flattening” problem. We’ll look at two different approaches. In both, the key will be to distinguish between handling elements that are lists (and therefore must be flattened recursively) and elements that are not lists. We’ll start from...

def flatten(L):

if __name__ == "__main__":
v = [[1,5], 6, [[2]], [3, [7, 8, [9,10], [11,12] ]]]
print(v)
print(flatten(v))

## Final Example: Merge Sort¶

• The fundamental idea of merge sort is recursive:

• Any list of length 1 is sorted
• Otherwise:
• Split the list in half
• Recursively sort each half
• Merge the resulting sorted halves
• We repeat our use of the merge function from Lecture 20:

def merge(L1,L2):
i1 = 0
i2 = 0
L = []
while i1<len(L1) and i2<len(L2):
if L1[i1] < L2[i2]:
L.append(L1[i1])
i1 += 1
else:
L.append(L2[i2])
i2 += 1
L.extend(L1[i1:])
L.extend(L2[i2:])
return L

• Using this, we will write the main merge_sort function in class.

def merge_sort(L):

• The solution will be posted on-line.
• We will also see how a minor mistake can cause the recursion to go into an infinite loop.
• Comparing what we write to our earlier non-recursive version of merge sort shows that the primary job of the recursion is to organize the merging process!

## Summary¶

• Functions that call themselves are known as “recursive functions”
• Use of a function call stack allows Python to handle recursive functions correctly.
• Many structures and functions important to computer science are defined recursively.
• Fundamentally, recurision is about defining a problem solution as a function of the solution to a simpler/shorter/smaller version of the problem.
• A basis case (or cases) is (are) always needed to make a recursion function succeed.
• Infinite recursion is avoided by ensuring that progress is made toward the basis case or cases in every recursive call.
• While many recursive functions are easily rewritten to remove the recursion, some advanced problems are difficult to solve without recursion.

1. Attempts to define the formal foundations of mathematics at the start of the 20th century depended heavily on recursion. While ultimately completing this formalization was shown to be impossible, the work led to much of the formal basis of computer science. Here is an example of the definition of adding two non-negative integers based on just +1 and -1 operations (formally known as successor and predecessor operations) and equality tests:

def add(m,n):
if n == 0:
return m
else:

Show the sequence of calls made by

print(add(5,3))

and then show the return values resulting from each recursive call.

2. Following the ideas from the previous problem, write a recursive function to multiply two non-negative integers using only the add function we’ve just defined, together with +1, -1 and equality tests.

3. Now, define the integer power function, $$\text{power}(x,n) = x^n$$, in terms of the mult function you just wrote, together with +1, -1, and equality.

4. Euclid’s algorithm for finding the greatest common divisor is one of the oldest known algorithms. If $$a$$ and $$b$$ are positive integers, with $$a \geq b$$, then let $$r$$ be the remainder of dividing $$a$$ by $$b$$. If $$r == 0$$, then $$b$$ is the GCD of the two integers. Otherwise, the GCD of $$a$$ and $$b$$ equals the GCD of $$b$$ and $$r$$. Here is the Python code:

def gcd(a,b):
if a < b:
a,b = b,a

r = a % b
if r==0:
return b
else:
return gcd(b,r)

1. What is the output of

print(gcd(36,24))
print(gcd(84,65))
print(gcd(84,66))

2. Why do we know that gcd is proceeding toward the base case (as required by our “rules” of writing recursive functions)?

5. Specify the recursive calls and return values from our merge_sort implementation for the list

L = [ 15, 81, 32, 16, 8, 91, 12 ]


text:
The ever growing need to efficiently store, retrieve and analyze massive datasets, originated by very different sources, is currently made more complex by the different requirements posed by users and applications. Such a new level of complexity cannot be handled properly by current data structures for big data problems.

To successfully meet these challenges, we launched a project, funded by the Italian Ministry of University and Research (PRIN no. 2017WR7SHH), that will lay down the theoretical and algorithmic-engineering foundations of a new generation of Multicriteria Data Structures and Algorithms. The multicriteria feature refers to the fact that we wish to seamlessly integrate, via a principled optimization approach, modern compressed data structures with new, revolutionary, data structures learned from the input data by using proper machine-learning tools. The goal of the optimization is to select, among a family of properly designed data structures, the one that “best fits” the multiple constraints imposed by its context of use, thus eventually dominating the multitude of trade-offs currently offered by known solutions, especially in the realm of Big Data applications.

# What is a multicriteria data structure?

A multicriteria data structure, for a given problem $P$, is defined by a pair $\langle \mathcal F, \mathcal A \rangle_P$ where $\mathcal F$ is a family of data structures, each one solving $P$ with a proper trade-off in the use of some resources (e.g. time, space, energy), and $\mathcal A$ is an optimisation algorithm that selects in $\mathcal F$ the data structure that best fits an instance of $P$.

For more details on the project, have a look at its full description here.

# Publications

Quickly discover relevant content by filtering publications.
On the performance of learned data structures. Theor. Comput. Sci., 2021.

Learning Based Compressed Data Structures (Poster). Stanford Compression Workshop, 2021.

The Alternating BWT: An algorithmic perspective. Theor. Comput. Sci., 812, 2020.

Learned data structures. Oneto L., Navarin N., Sperduti A., Anguita D. (eds) Recent Trends in Learning From Data. Studies in Computational Intelligence, vol 896. Springer, 2020.

# Software

#### BigRePair

A grammar compressor for huge files with many repetitions.

#### Block-epsilon tree

A compressed rank/select dictionary exploiting approximate linearity and repetitiveness.

#### CORENup

A Combination of Convolutional and Recurrent Deep Neural Networks for Nucleosome Positioning Identification.

An extensible framework to efficiently compute alignment-free functions on a set of large genomic sequences.

#### FASTdoopC

A general software framework for the efficient acquisition of FASTA/Q genomic files in a MapReduce environment.

#### FastKmer

A SPARK software system for the collection of $k$-mer statistics.

#### LA-vector

A compressed bitvector/container supporting efficient random access and rank queries.

#### Learned sorted table search

A software library to speed-up sorted table search procedures via learning from data.

#### PFP Data Structures

Data structures supporting Longest Common Extensions and Suffix Array queries, built on the prefix-free parsing of the text.

#### PGM-index

A data structure enabling fast searches in arrays of billions of items using orders of magnitude less space than traditional indexes.

#### PyGM

Python library of sorted containers with state-of-the-art query performance and compressed memory usage.

#### sHAM

Compression strategies and space-conscious representations for deep neural networks.

# Invited talks

Learning-based approaches to compressed data structures design
Extraction of Functional Knowledge from Large Biological Graphs and Applications in Precision Medicine
FM-index - the story... with a moral for (young) researchers
Introduction to compressed data structures
Learned Indexes
A rigorous approach to design learned data structures
On the compressed indexing of dictionaries of keys for massive key-value stores
A tutorial on learning-based compressed data structures
Theory and practice of learning-based compressed data structures
Learned and Compressed Data Structures, challenges on Storage Systems
Learned and Compressed Data Structures
The future of data structures: data-aware and self-designing
Life sciences and algorithmic design: speed and accuracy in small space
Hybrid Data Structures and beyond
The evolution of searching data structures

# Events

#### The future of compressed data structures, 20 years after the FM-index

July 19, 2022 – July 20, 2022 Lipari, Italy
Organised by Paolo Ferragina and Giovanni Manzini. Info

#### PhD course in Data compression and compressed data structures

May 16, 2022 – May 31, 2022 Pisa, Italy
Organised by Paolo Ferragina and Giovanni Manzini. Info

#### 2020 Student Challenge @ Algorithm Engineering UniPi course

November 4, 2020 – February 10, 2021 Pisa, Italy
Organised by Paolo Ferragina and Giorgio Vinciguerra. Info

# Meetings & Reports

#### Project end

August 28, 2022 (postponed to February 28, 2023 because of COVID-19)

#### Report of the second year

September 1, 2021

#### Report of the first year

September 1, 2020

#### Second meeting

February 6, 2020 – February 7, 2020 Dept. of Computer Science, Largo Bruno Pontecorvo 3, Pisa

#### Kickoff meeting

October 14, 2019 Video conference

#### Project start

September 1, 2019


text:
# Coming Soon

This problem is an extension of the problem “Wonderful Coloring – 1”. It has quite many differences, so you should read this statement completely.

Recently, Paul and Mary have found a new favorite sequence of integers a1,a2,,ana1,a2,…,an. They want to paint it using pieces of chalk of kk colors. The coloring of a sequence is called wonderful if the following conditions are met:

1. each element of the sequence is either painted in one of kk colors or isn’t painted;
2. each two elements which are painted in the same color are different (i. e. there’s no two equal values painted in the same color);
3. let’s calculate for each of kk colors the number of elements painted in the color — all calculated numbers must be equal;
4. the total number of painted elements of the sequence is the maximum among all colorings of the sequence which meet the first three conditions.

E. g. consider a sequence a=[3,1,1,1,1,10,3,10,10,2]a=[3,1,1,1,1,10,3,10,10,2] and k=3k=3. One of the wonderful colorings of the sequence is shown in the figure.

The example of a wonderful coloring of the sequence a=[3,1,1,1,1,10,3,10,10,2]a=[3,1,1,1,1,10,3,10,10,2] and k=3k=3. Note that one of the elements isn’t painted.
Help Paul and Mary to find a wonderful coloring of a given sequence aa.

Input

The first line contains one integer tt (1t100001≤t≤10000) — the number of test cases. Then tt test cases follow.

Each test case consists of two lines. The first one contains two integers nn and kk (1n21051≤n≤2⋅1051kn1≤k≤n) — the length of a given sequence and the number of colors, respectively. The second one contains nn integers a1,a2,,ana1,a2,…,an (1ain1≤ai≤n).

It is guaranteed that the sum of nn over all test cases doesn’t exceed 21052⋅105.

Output

Output tt lines, each of them must contain a description of a wonderful coloring for the corresponding test case.

Each wonderful coloring must be printed as a sequence of nn integers c1,c2,,cnc1,c2,…,cn (0cik0≤ci≤k) separated by spaces where

• ci=0ci=0, if ii-th element isn’t painted;
• ci>0ci>0, if ii-th element is painted in the cici-th color.

Remember that you need to maximize the total count of painted elements for the wonderful coloring. If there are multiple solutions, print any one.

Example

input

Copy
6
10 3
3 1 1 1 1 10 3 10 10 2
4 4
1 1 1 1
1 1
1
13 1
3 1 4 1 5 9 2 6 5 3 5 8 9
13 2
3 1 4 1 5 9 2 6 5 3 5 8 9
13 3
3 1 4 1 5 9 2 6 5 3 5 8 9

output

Copy
1 1 0 2 3 2 2 1 3 3
4 2 1 3
1
0 0 1 1 0 1 1 1 0 1 1 1 0
2 1 2 2 1 1 1 1 2 1 0 2 2
1 1 3 2 1 3 3 1 2 2 3 2 0

Note

In the first test case, the answer is shown in the figure in the statement. The red color has number 11, the blue color — 22, the green — 33.