text:
<img src="https://d5nxst8fruw4z.cloudfront.net/atrk.gif?account=iA1Pi1a8Dy00ym" style="display:none" height="1" width="1" alt="" />
You are reading an older version of this FlexBook® textbook: CK-12 Algebra - Basic Go to the latest version.

# 12.6: Addition and Subtraction of Rational Expressions

Difficulty Level: At Grade Created by: CK-12

Like numerical fractions, rational expressions represent a part of a whole quantity. Remember when adding or subtracting fractions, the denominators must be the same. Once the denominators are identical, the numerators are combined by adding or subtracting like terms.

Example 1: Simplify \begin{align*}\frac{4x^2-3}{x+5}+\frac{2x^2-1}{x+5}\end{align*}.

Solution: The denominators are identical; therefore we can add the like terms of the numerator to simplify.

Not all denominators are the same however. In the case of unlike denominators, common denominators must be created through multiplication by finding the least common multiple.

The least common multiple (LCM) is the smallest number that is evenly divisible by every member of the set.

What is the least common multiple of \begin{align*}2, 4x\end{align*}, and \begin{align*}6x^2\end{align*}? The smallest number 2, 4, and 6 can divide into evenly is six. The largest exponent of \begin{align*}x\end{align*} is 2. Therefore, the LCM of \begin{align*}2, 4x\end{align*}, and \begin{align*}6x^2\end{align*} is \begin{align*}6x^2\end{align*}.

Example 2: Find the least common multiple of \begin{align*}2x^2+8x+8\end{align*} and \begin{align*}x^3-4x^2-12x\end{align*}.

Solution: Factor the polynomials completely.

The LCM is found by taking each factor to the highest power that it appears in either expression. \begin{align*}LCM=2x(x+2)^2 (x-6)\end{align*}

Use this approach to add rational expressions with unlike denominators.

Example: Add \begin{align*}\frac{2}{x+2}-\frac{3}{2x-5}\end{align*}.

Solution: The denominators cannot be factored any further, so the LCM is just the product of the separate denominators.

The first fraction needs to be multiplied by the factor \begin{align*}(2x-5)\end{align*} and the second fraction needs to be multiplied by the factor \begin{align*}(x+2)\end{align*}.

We combine the numerators and simplify.

Combine like terms in the numerator.

## Work Problems

These are problems where two objects work together to complete a job. Work problems often contain rational expressions. Typically we set up such problems by looking at the part of the task completed by each person or machine. The completed task is the sum of the parts of the tasks completed by each individual or each machine.

Part of task completed by first person + Part of task completed by second person = One completed task

To determine the part of the task completed by each person or machine, we use the following fact.

Part of the task completed = rate of work \begin{align*}\times\end{align*} time spent on the task

In general, it is very useful to set up a table where we can list all the known and unknown variables for each person or machine and then combine the parts of the task completed by each person or machine at the end.

Example: Mary can paint a house by herself in 12 hours. John can paint a house by himself in 16 hours. How long would it take them to paint the house if they worked together?

Solution: Let \begin{align*}t=\end{align*} the time it takes Mary and John to paint the house together.

Since Mary takes 12 hours to paint the house by herself, in one hour she paints \begin{align*}\frac{1}{12}\end{align*} of the house.

Since John takes 16 hours to paint the house by himself, in one hour he paints \begin{align*}\frac{1}{16}\end{align*} of the house.

Mary and John work together for \begin{align*}t\end{align*} hours to paint the house together. Using the formula:

Part of the task completed = rate of work multiplied by the time spent on the task

we can write that Mary completed \begin{align*}\frac{t}{12}\end{align*} of the house and John completed \begin{align*}\frac{t}{16}\end{align*} of the house in this time and summarize the data in the following table.

Painter Rate of work (per hour) Time worked Part of Task
Mary \begin{align*}\frac{1}{12}\end{align*} \begin{align*}t\end{align*} \begin{align*}\frac{t}{12}\end{align*}
John \begin{align*}\frac{1}{16}\end{align*} \begin{align*}t\end{align*} \begin{align*}\frac{t}{16}\end{align*}

Using the formula:

Part of task completed by first person + Part of task completed by second person = One completed task

Write an equation to model the situation.

Solve the equation by finding the least common multiple.

## Practice Set

Sample explanations for some of the practice exercises below are available by viewing the following videos. Note that there is not always a match between the number of the practice exercise in the videos and the number of the practice exercise listed in the following exercise set.  However, the practice exercise is the same in both.

Perform the indicated operation and simplify. Leave the denominator in factored form.

1. \begin{align*}\frac{5}{24}-\frac{7}{24}\end{align*}
2. \begin{align*}\frac{10}{21}+\frac{9}{35}\end{align*}
3. \begin{align*}\frac{5}{2x+3}+\frac{3}{2x+3}\end{align*}
4. \begin{align*}\frac{3x-1}{x+9}-\frac{4x+3}{x+9}\end{align*}
5. \begin{align*}\frac{4x+7}{2x^2}-\frac{3x-4}{2x^2}\end{align*}
6. \begin{align*}\frac{x^2}{x+5}-\frac{25}{x+5}\end{align*}
7. \begin{align*}\frac{2x}{x-4}+\frac{x}{4-x}\end{align*}
8. \begin{align*}\frac{10}{3x-1}-\frac{7}{1-3x}\end{align*}
9. \begin{align*}\frac{5}{2x+3}-3\end{align*}
10. \begin{align*}\frac{5x+1}{x+4}+2\end{align*}
11. \begin{align*}\frac{1}{x}+\frac{2}{3x}\end{align*}
12. \begin{align*}\frac{4}{5x^2}-\frac{2}{7x^3}\end{align*}
13. \begin{align*}\frac{4x}{x+1}-\frac{2}{2(x+1)}\end{align*}
14. \begin{align*}\frac{10}{x+5}+\frac{2}{x+2}\end{align*}
15. \begin{align*}\frac{2x}{x-3}-\frac{3x}{x+4}\end{align*}
16. \begin{align*}\frac{4x-3}{2x+1}+\frac{x+2}{x-9}\end{align*}
17. \begin{align*}\frac{x^2}{x+4}-\frac{3x^2}{4x-1}\end{align*}
18. \begin{align*}\frac{2}{5x+2}-\frac{x+1}{x^2}\end{align*}
19. \begin{align*}\frac{x+4}{2x}+\frac{2}{9x}\end{align*}
20. \begin{align*}\frac{5x+3}{x^2+x}+\frac{2x+1}{x}\end{align*}
21. \begin{align*}\frac{4}{(x+1)(x-1)}-\frac{5}{(x+1)(x+2)}\end{align*}
22. \begin{align*}\frac{2x}{(x+2)(3x-4)}+\frac{7x}{(3x-4)^2}\end{align*}
23. \begin{align*}\frac{3x+5}{x(x-1)}-\frac{9x-1}{(x-1)^2}\end{align*}
24. \begin{align*}\frac{1}{(x-2)(x-3)}+\frac{4}{(2x+5)(x-6)}\end{align*}
25. \begin{align*}\frac{3x-2}{x-2}+\frac{1}{x^2-4x+4}\end{align*}
26. \begin{align*}\frac{-x^2}{x^2-7x+6}+x-4\end{align*}
27. \begin{align*}\frac{2x}{x^2+10x+25}-\frac{3x}{2x^2+7x-15}\end{align*}
28. \begin{align*}\frac{1}{x^2-9}+\frac{2}{x^2+5x+6}\end{align*}
29. \begin{align*}\frac{-x+4}{2x^2-x-15}+\frac{x}{4x^2+8x-5}\end{align*}
30. \begin{align*}\frac{4}{9x^2-49}-\frac{1}{3x^2+5x-28}\end{align*}
31. One number is 5 less than another. The sum of their reciprocals is \begin{align*}\frac{13}{36}\end{align*}. Find the two numbers.
32. One number is 8 times more than another. The difference in their reciprocals is \begin{align*}\frac{21}{20}\end{align*}. Find the two numbers.
33. A pipe can fill a tank full of oil in 4 hours and another pipe can empty the tank in 8 hours. If the valves to both pipes are open, how long would it take to fill the tank?
34. Stefan could wash the cars by himself in 6 hours and Misha could wash the cars by himself in 5 hours. Stefan starts washing the cars by himself, but he needs to go to his football game after 2.5 hours. Misha continues the task. How long did it take Misha to finish washing the cars?
35. Amanda and her sister Chyna are shoveling snow to clear their driveway. Amanda can clear the snow by herself in three hours and Chyna can clear the snow by herself in four hours. After Amanda has been working by herself for one hour, Chyna joins her and they finish the job together. How long does it take to clear the snow from the driveway?
36. At a soda bottling plant, one bottling machine can fulfill the daily quota in ten hours and a second machine can fill the daily quota in 14 hours. The two machines started working together but after four hours the slower machine broke and the faster machine had to complete the job by itself. How many hours does the fast machine work by itself?

Mixed Review

1. Explain the difference between these two situations. Write an equation to model each situation. Assume the town started with 10,000 people. When will statement b become larger than statement a?
1. For the past seven years, the population grew by 500 people every year.
2. For the past seven years, the population grew by 5% every year.
2. Simplify. Your answer should have only positive exponents. \begin{align*}\frac{16x^2 y^7}{-2x^8 y} \cdot \frac{1}{2} x^{-10}\end{align*}
3. Solve for \begin{align*}j: -12=j^2-8j\end{align*}. Which method did you use? Why did you choose this method?
4. Jimmy shot a basketball from a height of four feet with an upward velocity of 12 feet/sec.
1. Write an equation to model this situation.
2. Will Jimmy’s ball make it to the ten-foot-tall hoop?
5. The distance you travel varies directly as the speed at which you drive. If you can drive 245 miles in five hours, how long will it take you to drive 90 miles?
6. Two cities are 3.78 centimeters apart on an atlas. The atlas has a scale of \begin{align*}\frac{1}{2} cm=14 \ miles\end{align*}. What is the true distance between these cities?

8 , 9

Feb 22, 2012

Oct 19, 2015


text:
# Confusion regarding the Biot-Savart law

I was reading about the Biot-Savart law and there were two formulas stated in the textbook. One was stated as the Biot-Savart law

and the other was stated as the field due to a long, straight wire

My question is what is and why is there difference between two formulas.

format: plain


text:
# Air pressure relative to a force on a bag?

Assume an airtight bag occupied by air such that the pressure inside the bag is equal to the atmospheric pressure. Assume the surface tension of the bag is negligible.

What is the change in air pressure inside the bag relative to a weight placed on it? Is it possible to approximate the weight of the object on top of the bag by the air pressure inside the bag, without knowing the volume of the bag or the area of the object?

-

The air in the bag will be compressed when a weight is placed on the bag. The change of air pressure in the bag will change according to the bulk modulus of the fluid (air in this case) in the bag. The equation for bulk modulus is as follows: $$B=-\frac{\Delta p}{\Delta V/V}$$ In this case, $B$ is the bulk modulus of air, $\Delta p$ is the change of the air pressure inside the bag, and $V$ is the initial volume of the bag (before the weight was placed on the bag)

There are multiple ways to approximate the weight of the object on top of the bag. The choice of method to find the weight of the object depends on the information available. If we know $B$, $\Delta V$ and $V$, we could simply calculate $\Delta p$ by rearranging the equation for bulk modulus into $\Delta p = - B \frac{V}{\Delta V}$.

Now, let's consider two cases.

1. In the first case, the airtight bag is floating in the air. If the weight of the object is nonzero, the bag will accelerate downwards. If the density of the air inside the bag is equal to the density of the air outside the bag, the buoyant force acting upon the bag is zero because $F_B=\rho_{airoutsidebag} V_{bag}g$ (Archimedes' principle), and the weight of the bag is $F_g=m_{bag}g=\rho_{air in bag} V g$ since $\text{density}=\frac{\text{mass}}{\text{volume}} \rightarrow \rho = \frac{m}{V} \rightarrow m=\rho V$ where $\rho$ is density, $V$ is volume and $g$ is the acceleration due to gravity. (We assume that the bag is massless). Basically, in this case we would need very complex formulae and computations to calculate the flow of fluid around the bag since that would affect how much the bag compresses.

2. In the second case, the airtight bag is on the ground (or on some stationary, rigid object). For simplicity, assume that the airtight bag is actually a piston, and that the weight of the object on the airtight bag is the weight of the piston:

In this case, we can easily calculate the weight of the piston by the change of the volume of air in the system.

Let's say that the bottom of the piston has area $A$. Let $F_{air}$ be the force acting on the piston by the compressed air: $F_{air}=PA$. Let's assume that the piston system thing are just cuboids.Therefore, the initial volume of the air is $V_i=A h_i$ and the final volume of the air is $V_f=A h_f$. This gives us $\Delta V = V_f - V_i = A(h_f - h_i)$. Also, note that by definition, $V=V_i=a h_i$. Since $\Delta p = P_{atm}-P_{final}$, note that $\Delta p$ corresponds to the difference between the pressure inside and outside the piston system. Thus, $$F_{air}=A \Delta p=-AB \frac{v}{\Delta V}=-AB \frac{A h_i}{A (h_f - h_i)}=AB \frac{h_i}{h_i - h_f}$$

At equilibrium, there is no acceleration. Thus, $F_{up}$ is equal to the weight of the object on the airtight bag, which I refer to as $F_g$. In other words, $F_{up}=F_g$, giving us a formula for the weight of the object:

$$F_g=AB \frac{h_i}{h_i - h_f}$$

Notice how I had to make so many simplifying assumptions in order to make the calculations simple. If we used complex shapes, in other words if we tried to compute the weight of the object in a more realistic situation, we would need very complex fluid dynamics calculations which depend on many factors, such as the material used to make the airtight bag and the viscosity of the air.

-
So in case 2, if I know the B, dp, and V, I can calculate dV (change in volume). But how do I calculate the weight of the object from dV? –  RobB May 4 '13 at 20:42
@RobB I edited my answer to add that information. See physics.stackexchange.com/posts/63304/revisions for side-by-side comparison of the edit. –  raindrop May 5 '13 at 5:58


text:
# Would the time dependent wave equation of a free particle be the same in 4D Euclidean spacetime as it is in 4D Minkowski spacetime?

In 4d minkowski spacetime the time dependent wave equation of a free particle is $$\Psi(x,t) = \left({a \over a + i\hbar t/m}\right)^{3/2} e^{- {x^2\over 2(a + i\hbar t/m)} }.$$ I notice that there is the imaginary unit in the equation, and in minkowski spacetime space can be thought of as imaginary time, and time can be thought of as imaginary space. In euclidean spacetime the minus sign in the equation for the spacetime interval $${{\Delta}s}^2={{\Delta}x}^2+{{\Delta}y}^2+{{\Delta}z}^2-(c{\Delta}t)^2$$ would get replaced by a plus sign, so that the spacetime interval between two events would be $${{\Delta}s}^2={{\Delta}x}^2+{{\Delta}y}^2+{{\Delta}z}^2+{{\Delta}w}^2.$$ Is the imaginary unit in the time dependent wave equation of a free particle related to time being imaginary space, and space being imaginary time? The time dependent wave equation is time symmetric, but if I simply replace the imaginary unit with 1, I do not get a time symmetric equation.

In a 4d euclidean spacetime, in which every spacetime coordinate of an event was always a real number, would the wave equation be the same as in 4d minkowski spacetime, and have the imaginary unit?


text:
# Why the same value of float and a string that is converted into float using function atof are not equal?

I have written a program in C and when I compare the same values of a float and a string that is converted into float using function atof results in NOT EQUAL .

    #include<stdio.h>

main(){

char str[10] = "54.23" ;
float val = 54.23 ;

if( atof(str) == val )
printf("\nconverted correctly");
else
printf("\nThen What is the use of atof\n ");

}

This Program is showing output : "Then What is the use of atof" Please tell me why this anonymous behavior is shown by this program ?

-
Uhm, val is an int in your example... –  Anders Lindahl Aug 5 '12 at 7:03
Sorry while i am checking it i change float to int because i m trying it with atoi funtion. Please modify "int" to "float" . It is alsi not working with float. –  Ajay Yadav Aug 5 '12 at 7:16
Other than the name might suggest atof returns a double. Other than the other answers suggest, if you use the correct type there should be no difference. –  Jens Gustedt Aug 5 '12 at 7:35
Basically, the question changed its nature since posted. Let's close it. –  Jens Gustedt Aug 5 '12 at 7:37
#include <stdlib.h> –  Rsh Aug 5 '12 at 7:53

## Never test floats/doubles for equality with ==

Here's a version of your code which actually displays the values in question:

#include <stdio.h>
#include <stdlib.h>

int main()
{
char str[10] = "54.23";
float val = 54.23;

printf("atof(str) = %.15f\n", atof(str));
printf("      val = %.15f\n", val);

return 0;
}

When you run it you see this:

$gcc -Wall atof.c$ ./a.out
atof(str) = 54.229999999999997
val = 54.229999542236328
\$

The values are close, within the expected accuracy of a single precision float, but they are not identical. Also, as others have noted, atof returns a double, so you are comparing the value of a float promoted to a double with a full precision double as returned by atof.

As always with this type of question, read this before proceding any further with floating point arithmetic in your code. The "take home message" is that you should never compare floats or doubles with == - always compare the absolute difference with an appropriate tolerance value.

-
“Always compare the absolute difference with a tolerance value” is poor advice because it solves the problem of false negatives by creating a problem of false positives. There is not a simple single solution here; users of floating-point should understand it and be thoughtful about the operations they use. –  Eric Postpischil Aug 5 '12 at 10:37

Because val is an int; when you assign it 54.23 it'll be truncated to 54. And 54 != 54.23.

Even if it was a float, you couldn't expect them to be equal. Here's why.

-
It is also not working while i am trying after changing int to float . You can give it a try. –  Ajay Yadav Aug 5 '12 at 7:17
@AjayYadav yes, it's also not working, and it's expected. Really, click the link I made in my answer. –  user529758 Aug 5 '12 at 7:32
double atof(char *str);\\it return double not a float

This comparison is between a float and a double.As you compare between two different types you may get some unexpected output.because every data type having different memory representation as well as different access mechanism.

float represent in memory in different form as compare to double .

http://en.wikipedia.org/wiki/Floating_point#Internal_representation

Again you should include the header file

#include <stdlib.h> \\prototype of atof() present in this header.

if you not provide the proper prototype before use of the function then

return type of function by default int .So I think the return result is definitely different as you expected.

-

You almost never want to check equality with floating point numbers, because teensy differences will be read as unequal. There are other problems, too. For example, even if you use double precision, for instance, the decimal number "0.1" is represented as "0.10000000000000001".

In this case, your "val" variable is a double precision literal, which is cast to a float. The result probably won't be accurate perfectly. Secondly, your string literal needs to convert from base ten to a base 2 double. So, to compare the atof value to your literal, atof converts a base ten string to a base two double, while "val" was converted from a base ten literal to a base two double to a base two float, and then upcast back to a base two double to do the comparison.

Point of fact, I'm not going to pin down exactly where that lost precision went. Do as Paul's code might suggest and compare the values to within a tolerance.

-
There is no need to throw up one’s hands and give up understanding where errors occur. These things are reasonably well specified and understood. First, “54.23” in the source text is converted to a double. Any good compiler will convert it to the value representable as a double that is closed to the number 54.23, although the C standard allows a little slack. Then, when val is initialized, that double is converted to a float. With any implementation using IEEE 754, the result is the value representable in float that is nearest the double. Converting val from float to double has no error. –  Eric Postpischil Aug 5 '12 at 13:06


text:
# How can one approximately guesstimate wave drag?

Wave drag is an especially high drag that affects supersonic vehicles above mach 1, but I've found it hard to find source materials to estimate wave drag. I'm looking at the transonic region till Mach 1.2 and so I'm looking to have an estimate about how I can design better for wave drag by estimating how it would affect my flight in the region mach 0.8 - 1.2.

• Welcome to Aviation.SE. Speeds up to Mach 1 are subsonic by definition and do not have wave drag. You might want to simplify your question to focus on one aspect. It requires several answers in its current form, and this site tends to like one at a time. – Pilothead Sep 18 '18 at 18:15
• Hey @Pilothead, thanks for the heads up. Thanks for welcoming me , and I have simplified the question and perhaps I'll add the other elements in another question. – Rajath Pai Sep 19 '18 at 1:28

Usually wave drag $$c_{d_w}$$ starts becoming relevant at the drag divergence Mach number $$M_{DD}$$, which is defined as

$$\frac{\partial c_{d_w}}{ \partial M}_{M=M_{DD}}=0.1 \tag{1}$$

Or in words, the Mach number at which the wave drag coefficient versus M has a gradient of 0.1

Lock$$^1$$ has derived an empirical formula that shows the development of $$c_{d_w}$$ after the critical Mach number, $$M_{cr}$$, which is the number at which the flow becomes locally supersonic on the wing:

$$c_{d_w} = 20(M-M_{cr})^4 \tag{2}$$

If we apply the definition of $$(1)$$ to $$(2)$$ we obtain:

$$M_{cr} = M_{DD} - \left(\frac{0.1}{80}\right)^{1/3} \tag{3}$$

As you can see, the value of $$M_{DD}$$, is a little bit higher than the value of $$M_{cr}$$. First you get local super sonic flow, but the drag does not increase rapidly, as you further increase the Mach number, the drag starts to increase significantly, and you reach $$M_{DD}$$. See this image, where $$M_{DD}$$ is denoted by $$M_{CDR}$$.

Source

The value of $$M_{DD}$$ can be estimated using:

$$M_{DD} = \frac{K_A}{cos \Lambda} + \frac{t/c}{cos^2 \Lambda} +\frac{c_l}{cos ^3\Lambda} \tag{4}$$

With $$\Lambda$$ is the sweep angle, $$t/c$$ the relative thickness, $$c_l$$ the lift coefficient, and $$K_A$$ the airfoil technology factor (a correction factor denoting the quality of the airfoil).

So the workflow is as follows:

1. Estimate $$M_{DD}$$ using $$(4)$$
2. Derive $$M_{cr}$$ using $$(3)$$
3. Determine the development of $$c_{d_w}$$ using $$(2)$$.

$$^1$$ Hilton, W.F., High Speed Aerodynamics, Longmans, Green & Co., London, 1952, pp. 47-49


text:
Thesis

### Interaction of molecular Rydberg states with metal surfaces

Abstract:

The interaction between high-n Rydberg states of molecular hydrogen and metal surfaces has been investigated for the first time. Rydberg states of hydrogen possessing either 0 or 2 units of rotational angular momentum, defined by the quantum number N+ , and principal quantum numbers in the range n= 17 22 (for the N+= 2 states) and n=41-45 (for the N+= 0 states) are directed at a grazing angle onto a met...

### Access Document

Files:
• (pdf, 24.9MB)

### Authors

More by this author
Institution:
University of Oxford
Division:
MPLS
Role:
Author

#### Contributors

Role:
Supervisor
Role:
Supervisor
Publication date:
2005
Type of award:
DPhil
Level of award:
Doctoral
Awarding institution:
University of Oxford
Source identifiers:
603851770
Language:
English
Subjects:
UUID:
uuid:7467904f-96c1-4a10-856e-d835375cf762
Local pid:
td:603851770
Deposit date:
2013-01-21


text:
# Taylor Functions – explanation and software tools

• Taylor functions are used in areas such as power engineering, fluid mechanics and bathymetry to simplify often complex functions into more understandable terms.
• Manually calculating Taylor functions is mind blowing - thankfully there are an array of software tools to do this.
• Where some exact elements of an equation are not available, Taylor functions allow an improvement on traditional approximation calculations.

The Taylor Functions are series of sum terms that concern the values of the derivatives of a given function at a given point, so they are a part of Calculus. They are a clever practical way to expand and simplify complicated and transcendental functions into computable polynomials. The number of the sum terms that are used to represent the specific function is theoretically infinite, so we are either following an “adequate approximation” approach or we use software to calculate the series of the sum using a large number of terms.

## Taylor Functions in practice

Let’s suppose that we want to calculate the function ex for x=2. Euler’s number is an irrational number which means that its digits are infinite, so our calculation will be approaching a value of 2.7182 = 7.387524.

Now if we use Taylor’s Functions, we first consider the derivative of ex which happens to be ex. So, for our series, we will get: “Σ (from 0 to n) xn/n!” which translates to: (x0)+(x1/1!) + (x2/2!) + (x3/3!) +… (xn/n!). For practical purposes, I will showcase the calculation of the series for the first five terms which are: (20)+(21/1)+[22/(1+2)]+[23/(1+2+3)]+[24/(1+2+3+4)]+[25/(1+2+3+4+5)] = 1+2+2+1.333+0.667+0.267 = 7.267. Now it is obvious that this value is not close enough to e2, but this is because we haven’t used enough terms in our series of sums. So, if we continue calculating the sum to up to 10 terms the value becomes 7.3889, and once we get to 20 terms it becomes 7.389056.

## What is their use in Engineering

Most real things and systems in nature, engineering practice and phenomena are complex, at least to our mind. This means that their mathematical representation is equally complex making things harder for us to calculate. Taylor Functions are used exactly for the purpose of simplifying things to make them easier to work with. Here are a couple of examples of where Taylor Functions are used:

• In power engineering, the flow of electric power in an interconnected system involves many variables that are initially unknown. To address this, engineers use the Newton-Raphson solution method that is based on the use of Taylor Functions.
• Fluid mechanics engineers use the Taylor series in conjunction with the Navier-Stokes equation to achieve an accurate calculation method when studying arbitrary shapes with the Galerkin Computational method.
• Similarly, bathymetry studies involve Taylor Functions based on the Dirichlet-Neumann operator to represent three-dimensional wave motion in two-dimensional mathematics (Hamiltonian).
• Taylor functions are also used in error propagation studies as an alternative to the Monte-Carlo method.
• Harmonic oscillation systems can be analysed more simply and essentially better understood through the use of Taylor Functions.
• Engineering and physics simulation software such as ANSYS, MATLAB, and Abaqus use Taylor functions to calculate heat and mass transfer, fluid dynamics, etc.

## Software tools

Because calculating Taylor Functions is only for psychopaths, doing so on a software tool is standard practice for an engineer nowadays. Here are some common online and offline tools that you can use for the calculation of Taylor Functions:

Online

Offline


text:
# AP Calculus BC Monday, 18 November 2013 OBJECTIVE TSW use integration by subsitution. ASSIGNMENT DUE –Sec. 4.1  wire basket Tests are graded. Look on.

## Presentation on theme: "AP Calculus BC Monday, 18 November 2013 OBJECTIVE TSW use integration by subsitution. ASSIGNMENT DUE –Sec. 4.1  wire basket Tests are graded. Look on."— Presentation transcript:

AP Calculus BC Monday, 18 November 2013 OBJECTIVE TSW use integration by subsitution. ASSIGNMENT DUE –Sec. 4.1  wire basket Tests are graded. Look on line for your score. –Due to time constraints, we will not go over them in class. –If you would like to look at your test, arrange to come in during lunch, before school, or after school. The test for indefinite integration has been pushed back to this Friday, 22 November 2013.

Sec. 4.5: Integration by Substitution

Rewrite the integral:

Sec. 4.5: Integration by Substitution u = 7x du = 7 dx

Sec. 4.5: Integration by Substitution u = x 2 – 4 du = 2x dx 1/2 du = x dx

Sec. 4.5: Integration by Substitution u = sin 3x du = (cos 3x) 3 dx 1/3 du = cos 3x dx

Sec. 4.5: Integration by Substitution u = 5x + 3 du = 5 dx 1/5 du = dx

Sec. 4.5: Integration by Substitution Rearranging u u = x + 3 du = dx u – 3 = x

Sec. 4.5: Integration by Substitution u = 2x – 1 du = 2 dx u + 1 = 2x 1/2 du = dx (u + 1) / 2 = x

Download ppt "AP Calculus BC Monday, 18 November 2013 OBJECTIVE TSW use integration by subsitution. ASSIGNMENT DUE –Sec. 4.1  wire basket Tests are graded. Look on."

Similar presentations


text:
# The other side of the moon

/bb|[^b]{2}/
Never stop Grokking

## Monday, December 27, 2010

### Using bandwidth to mitigate latency

[This post is mirrored from the Performance Advent Calendar]

The craft of web performance has come a long way from yslow and the first few performance best practices. Engineers the web over have made the web faster and we have newer tools, many more guidelines, much better browser support and a few dirty tricks to make our users' browsing experience as smooth and fast as possible. So how much further can we go?

#### Physics

The speed of light has a fixed upper limit, though depending on the medium it passes through, it might be lower. In fibre, this is about 200,000 Km/s, and it's about the same for electricity through copper. This means that a signal sent over a cable that runs 7,000Km from New York to the UK would take about 35ms to get through. Channel capacity (the bit rate), is also limited by physics, and it's Shannon's Law that comes into play here.

This, of course, is the physical layer of our network. We have a few layers above that before we get to TCP, which does most of the dirty work to make sure that all the data that your application sends out actually gets to your client in the right order. And this is where it gets interesting.

#### TCP

Éric Daspet's article on latency includes an excellent discussion of how slow start and congestion control affect the throughput of a network connection, which is why google have been experimenting with an increased TCP initial window size and want to turn it into a standard. Each network roundtrip is limited by how long it takes photons or electrons to get through, and anything we can do to reduce the number of roundtrips should reduce total page download time, right? Well, it may not be that simple. We only really care about roundtrips that run end-to-end. Those that run in parallel need to be paid for only once.

When thinking about latency, we should remember that this is not a problem that has shown up in the last 4 or 5 years, or even with the creation of the Internet. Latency has been a problem whenever signals have had to be transmitted over a distance. Whether it is a rider on a horse, a signal fire (which incidentally has lower latency than light through fibre[1]), a carrier pigeon or electrons running through metal, each has had its own problems with latency, and these are solved problems.

#### C-P-P

There are three primary ways to mitigate latency. Cache, parallelise and predict[2]. Caching reduces latency by bringing data as close as possible to where it's needed. We have multiple levels of cache including the browser's cache, ISP cache, a CDN and front-facing reverse proxies, and anyone interested in web performance already makes good use of these. Prediction is something that's gaining popularity, and Stoyan has written a lot about it. By pre-fetching expected content, we mitigate the effect of latency by paying for it in advance. Parallelism is what I'm interested in at the moment.

#### Multi-lane highways

Mike Belshe's research shows that bandwidth doesn't matter much, but what interests me most is that we aren't exploiting all of this unused channel capacity. Newer browsers do a pretty good job of downloading resources in parallel, and with a few exceptions (I'm looking at you Opera), can download all kinds of resources in parallel with each other. This is a huge change from just 4 years ago. However, are we, as web page developers, building pages that can take advantage of this parallelism? Is it possible for us to determine the best combination of resources on our page to reduce the effects of network latency? We've spent a lot of time, and done a good job combining our JavaScript, CSS and decorative images into individual files, but is that really the best solution for all kinds of browsers and network connections? Can we mathematically determine the best page layout for a given browser and network characteristics[3]?

#### Splitting the combinative

HTTP Pipelining could improve throughput, but given that most HTTP proxies have broken support for pipelining, it could also result in broken user experiences. Can we parallelise by using the network the way it works today? For a high capacity network channel with low throughput due to latency, perhaps it makes better sense to open multiple TCP connections and download more resources in parallel. For example, consider these two pages I've created using Cuzillion:
Have a look at the page downloads using FireBug's Net Panel to see what's actually happening. In all modern browsers other than Opera, the second page should load faster whereas in older browsers and in Opera 10, the first page should load faster.

Instead of combining JavaScript and CSS, split them into multiple files. How many depends on the browser and network characteristics. The number of parallel connections could start of based on the ratio of capacity to throughput and would reduce as network utilisation improved through larger window sizes over persistent connections. We're still using only one domain name, so no additional DNS lookup needs to be done. The only unknown is the channel capacity, but based on the source IP address and a geo lookup[4] or subnet to ISP map, we could make a good guess. Boomerang already measures latency and throughput of a network connection, and the data gathered can be used to make statistically sound guesses.

I'm not sure if there will be any improvements or if the work required to determine the optimal page organisation will be worth it, but I do think it's worth more study. What do you think?

#### Footnotes

1. Signal fires (or even smoke signals) travel at the speed of light in air v/s light through fibre, however the switching time for signal fires is far slower, and you're limited to line of sight.
2. David A. Patterson. 2004. Latency lags bandwith[PDF]. Commun. ACM 47, 10 (October 2004), 71-75.
3. I've previously written about my preliminary thoughts on the mathematical model.
4. CableMap.info has good data on the capacity and latency of various backbone cables.

## Thursday, December 02, 2010

### Bad use of HTML5 form validation is worse than not using it at all

#### ...or why I'm glad we don't use fidelity any more

Fidelity's online stock trading account uses HTML5 form's pattern attribute to do better form validation, only they get it horribly wrong.

First some background...

The pattern attribute that was added to input elements allows the page developer to tell the browser what kind of pre-validation to do on the form before submitting it to the server. This is akin to JavaScript based form validation that runs through the form's onsubmit event, except that it's done before the form's onsubmit event fires. Pages can choose to use this feature while falling back to JS validation if it isn't available. They'd still need to do server-side validation, but that's a different matter. Unfortunately, when you get these patterns wrong, it's not possible to submit a valid value, and given how new this attribute is, many web devs have probably implemented it incorrectly while never having tested on a browser that actually supports the attribute.

This is the problem I faced with fidelity. First some screenshots. This is what the wire transfer page looks like if you try to transfer $100. The message shows up when you click on the submit button: On Firefox 4: On Opera: There are a few things wrong with this. First, to any reasonable human being (and any reasonably well programmed computer too), the amount entered (100.00) looks like a valid format for currency information. I tried alternatives like$100, $100.00, 100, etc., but ended up with the same errors for all of them. Viewing the source told me what the problem was. This is what the relevant portion of the page source looks like: $<input type="text" name="RED_AMT" id="RED_AMT"
maxlength="11" size="14" value="" tabindex="10"

#### MacOSX

On the Mac, it's stored in
~/Library/Application Support/Firefox/Profiles/<profile>/extensions
/Library/Application Support/Mozilla/Extensions

## Wednesday, August 04, 2010

### Site theme

Ten months down the line,
I still like this site's style,
I've tweaked it a bit,
That bluesmoon did this one fine.

When I work on a site, I always start with the design first. I'm a visual developer, so I really need to see what things will look like when I'm done. I also need to know if I can stand the design enough to stare at it for days on end during development. The result is that I end up hating my design in a couple of days and change things around, most often it's the colours.

This has happened a lot with earlier versions of my site's style, but the version I have now is one I like. I first pushed it out standardised across all my subdomains back in November 2009, and now 10 months down the line, it still pleases me.

I like the colours. They don't shout out at me, but frame my content well. Important content is visible while unimportant meta information is dimmed out. I played around with the margins a lot to get it to work the way I want across devices. It's not ideal, but it gets the job done. I took inspiration for the fonts from several sites until I settled on what I use now.

There's still a few issues. Not everyone likes the tag cloud, but it's the best way for me to navigate my site, especially since google's site search doesn't seem to work. The home page looks terrible on my E71, and I should get to fixing that some day, but other than that, it's ok.

## Monday, August 02, 2010

### 4.01 Strict — Tennis Elbow

My left elbow's out of commission, so there will probably not be a strip until it gets better. The technical term is Lateral Epicondylitis, commonly known as Tennis Elbow. What's ironic is that I don't play Tennis. It happened due to bad typing posture.

## Thursday, July 29, 2010

### Boomerang and the WebTiming API

I've been doing a lot of development on boomerang over the last few weeks. In particular around the WebTiming API. During Velocity this year, Microsoft announced that their latest release of IE9 supported this API, so I set about adding support to boomerang.

Since then, Google have announced support in Chrome as well, and there are rumours around that Mozilla plans on adding it to firefox. Now since the specification is still a draft, none of these browsers use the window.performance object, but instead use their own browser specific version. I've now added support for all these versions into boomerang.

More details in my post on the YDN blog.

You can get boomerang from github, and join the discussion on our forum.

## Monday, July 26, 2010

### 4.01 Strict — Invention of the Web

Tim Berners-Lee invents the WWW

There was a young man named Tim,
Who wanted to share docs on a whim,
He built HTTP,
But the W3C
Made standards so webdevs would sin

## Sunday, July 25, 2010

### Handling Date/Times and timezones

Wrote this as an answer to a question on StackOverflow, but I think it was one of my better ones, so am reproducing it here.

There are four different times you should consider:
1. Event time: eg, the time when an international sporting event happens, or a coronation/death/etc. This is dependent on the timezone of the event and not of the viewer.
2. Television time: eg, a particular TV show is broadcast at 9pm local time all around the world. Important when thinking about publishing the results (of say American Idol) on your website
3. Relative time: eg: This question has an open bounty closing in 21 hours. This is easy to display
4. Recurring time: eg: A TV show is on every Monday at 9pm, even when DST changes.
There is also Historic/alternate time. These are annoying because they may not map back to standard time. Eg: Julian dates, dates according to a Lunar calendar on Saturn, The Klingon calendar.

Storing start/end timestamps in UTC works well. For 1, you need an event timezone name + offset stored along with the event. For 2, you need a local time identifier stored with each region and a local timezone name + offset stored for every viewer (it's possible to derive this from the IP if you're in a crunch). For 3, store in UTC seconds and no need for timezones. 4 is a special case of 1 or 2 depending on whether it's a global or a local event, but you also need to store a created at timestamp so you can tell if a timezone definition changed before or after this event was created. This is necessary if you need to show historic data.

#### Storing times

• Always store time in UTC
• Convert to local time on display (local being defined by the user looking at the data)
• When storing a timezone, you need the name, timestamp and the offset. This is required because governments sometimes change the meanings of their timezones (eg: the US govt changed DST dates), and your application needs to handle things gracefully... eg: The exact timestamp when episodes of LOST showed both before and after DST rules changed.
##### Offsets and names
An example of the above would be:

The soccer world cup finals game happened in South Africa (UTC+2--SAST) on July 11, 2010 at 19:00 UTC.

With this information, we can historically determine the exact time when the 2010 WCS finals took place even if the South African timezone definition changes, and be able to display that to viewers in their local timezone at the time when they query the database.
System Time

You also need to keep your OS, database and application tzdata files in sync, both with each other, and with the rest of the world, and test extensively when you upgrade. It's not unheard of that a third party app that you depend on did not handle a TZ change correctly.

Make sure hardware clocks are set to UTC, and if you're running servers around the world, make sure they're OSes are configured to use UTC as well. This becomes apparent when you need to copy hourly rotated apache log files from servers in multiple timezones. Sorting them by filename only works if all files are named with the same timezone. It also means that you don't have to do date math in your head when you ssh from one box to another and need to compare timestamps.

Also, run ntpd on all boxes.

#### Clients

Never trust the timestamp you get from a client machine as valid. For example, the Date: HTTP headers, or a javascript Date.getTime() call. These are fine when used as opaque identifiers, or when doing date math during a single session on the same client, but don't try to cross-reference these values with something you have on the server. Your clients don't run NTP, and may not necessarily have a working battery for their BIOS clock.

#### Trivia

Finally, governments will sometimes do very weird things:
Standard time in the Netherlands was exactly 19 minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through 1937-06-30. This time zone cannot be represented exactly using the HH:MM format.

## Monday, July 19, 2010

### 4.01 Strict — Don't mess with Stubbornella

Mixing CSS is pixel princess, but weigh her down, and you'll have to deal with... Stubbornella.

## Monday, July 12, 2010

### 4.01 Strict — SpeedFreak

Mild mannered Stoyan Stefanov makes an appearance in this week's 4.01 Strict.

## Tuesday, July 06, 2010

### 4.01 Strict — The discovery of JSON

In episode 2, Douglas Crockford discovers JSON

## Monday, June 28, 2010

### My velocity talk

So I was supposed to speak at Velocity 2010 this year. My talk was titled Latency: Why you should worry and what you can do about it. It was one of the last few talks at the conference. Unfortunately though, the thing about a conference like velocity is that almost everything is about latency, and by the time I went on stage, absolutely every point that I'd planned on making had already been covered.

That's not a bad thing. For one, it validates my ideas, telling me that many other smart people have been thinking about the same things. Secondly, each of these speakers had 60-90 minutes to convey their ideas, which meant they could really do it justice. It also shows that velocity is a pretty awesome conference to be at.

So, by the night of the 23rd, I realised that there was no way I could go ahead with my talk in its current state, and sat down to rewrite it. That's when the drama started.

Now my usual format is to build my slides using LaTeX and generate a PDF from that. I have a bunch of relevant images and charts in there, and it's not too hard to do that with LaTeX. I started going down that path, but realised that I wanted to do something different. I really wanted to reference all the other presentations and go over points that they'd covered, so I switched to Comic Life. I was up until 5am, working out the details and getting the script right.

The thing about a 12 minute talk is that you can't waste a single minute thinking about what you're going to say next. You just need to know it, and there wasn't much time for me to prepare. I'd also planned to announce boomerang in the latter part of my talk, so had to fit that in as well.

At 5 I realised that I was too sleepy to keep going, so I went to bed and figured I could finish things off in a couple of hours once I woke up. I was also too sleepy to backup the presentation to my server.

I woke up at 10, and opened up the mac to start work again, and it wouldn't come on. I could hear the fans whirring inside, and the startup sound would sound, but the screen wouldn't turn on. It was now officially time to panic.

Two things were in my favour though. I work at Yahoo!, and the conference was just down the road from Yahoo! HQ. I headed in to the office and took the mac down to IT. The guys there were really helpful. They opened up a new Mac, and transferred my hard disk to it. The new one started up perfectly, and it's what I'm using right now.

I spent the next two hours putting the finishing touches on, and adding details for the boomerang release, and then rushed off to the conference with a slice of pizza in my hand.

Once I got on stage, I plugged in the mac, and as it has done on numerous occasions in the past, it hung as soon as the VGA adaptor was plugged in. I had to reboot while on stage, and this was all coming out of the 12 minutes that I had.

The talk went off okay. Looking back at the video, there's a bunch of things I'd have done differently, and some more details that I'd have covered, but I'll let you decide for yourselves.

My slides are on slideshare, while the video is on blip.tv.

### 4.01 Strict — I am codepo8

I've been playing around with comic life, and this is what I ended up with:

I'm calling it 4.01 Strict.

## Saturday, June 26, 2010

### boomerang

I've been silent on this blog for a while, and the main reason for that is boomerang. It's taken a few months to get it ready for a public release, and I'm quite pleased with the way it's come out.

As many of you know, I've worked with the Exceptional Performance team at Yahoo! for over two years, and during this time, I've done a lot of work with measuring and analysing the performance of all of Yahoo!'s websites (yes, all). There are several tens of thousands of page types, and probably hundreds of thousands of unique URLs that could show up on a yahoo.com domain. This throws up some very interesting problems that I'm not sure I'd have had the chance to work with anywhere else.

I learnt a lot about scaling cheap hardware to meet the needs of this system, and also a lot about how each of these thousands of pages worked to improve their performance.

The measurement tool that we use has turned into boomerang. It's primarily a beaconing framework, so isn't restricted to performance data, but the core plugins that come packaged with it only do performance measurement.

I'll write more in later posts about what you can do with boomerang, but for now, go get it from github.com/yahoo/boomerang. Fork it, use it, patch it, tell us what you think of it.

There's also documentation along with the source code, or you can have a look at my personal copy of them — frequently, but not automatically updated.

I announced it at Velocity 2010, and they made a video of my talk.

## Thursday, June 10, 2010

### location.href

While working on boomerang, I needed to get at the current document's URL. The way I generally do that is via the location.href property. This worked perfectly until I hit upon a URL with a # in it. The # was correctly URL encoded to %23, and most browsers returned the correct string in location.href.

Safari 4.x was the only exception. It returned the URL after decoding. This wouldn't be a problem on its own, except that only the URL portion was decoded. location.search (everything after the ?) and location.hash (everything after the #) were untouched.

This meant that a URL like this:
http://tech.bluesmoon.info/foobar%231.html#123

Would show up in location.href as this:
http://tech.bluesmoon.info/foobar#1.html#123

This was a problem, because I now had two # signs in the URL, and would have to twist through hoops to correctly encode the URL. The bug is actually in Webkit, and affects Android too. Thanks to Ryan Grove for pointing me to it.

Then I discovered the document.URL property. This is exactly the same as location.href, except that it isn't affected by the bug in Safari 4.x.

I've tested in other browsers, and it seems to work as expected, so that's what I'm using for now.

Oh, and if you don't know what boomerang is yet, wait until June 24.

## Tuesday, April 13, 2010

### Internet access from Mumbai Airport's Clipper Lounge

If you're a frequent flier, or are flying first or business class out of Mumbai, then you'll get access to the Clipper Lounge. They provide free wifi, but it doesn't really work as advertised.

You first connect to the wifi network, and open your browser, which gets redirected to a login page. You click on Complimentary Internet access, and get to a form that asks for your personal details, and phone number, claiming that they will send you an access code via SMS.

Do NOT enter your real phone number here. For starters, no SMS gets sent. Secondly, you'll start getting SMS spam. Use any 10 digit phone number. Enter 91 for the country code.

Now, once you do this, you'll get an intermediate page that will immediately redirect to a second page that shows you your username and password. The password on that page is wrong. Instead, what you want to do is hit escape as soon as you get the intermediate page. It's blue, and it has the database insert statement that looks like this:
insert into users
u_current_quota,u_persistant_quota,u_interim_interval,
u_idle_timeout,u_expiration_date,u_nbr_concurrent_sessions,
u_max_concurrent_sessions, u_nbr_starts, u_nbr_stops,
fk_group_id,fk_profile_id
) values (
'cc3zo7s', '2497161', '2', 'Morons',
'86400', '86400', '1', '15', '600',
'2010-05-13 22:51:28','0','1',
'0','0','2','2'
);

In the above string, cc3zo7s is the username and 2497161 is the password that you need to use.

Note, that if you enter a phone number that already exists in the database (eg: one of the tech support numbers on the login page), you'll get this instead:
Duplicate entry '1271179237' for key 'PRIMARY'

Just try a different phone number. Oh, and what's with the semi-colon at the end of the SQL statement?

Lastly, don't bother sending them feedback. It doesn't go anywhere. And I don't mean that it goes into some mailbox that no one reads. I mean it goes nowhere. This is the page you get when you try to send feedback:

The interesting text in there is:
Warning: mail() [function.mail]: Failed to connect to mailserver at
"ezcommindia.com" port 25, verify your "SMTP" and "smtp_port" setting in
php.ini or use ini_set() in C:\xampp\htdocs\ezcomm\ifeedback.php
on line 225

Yup, real classy. I think we can just start from the end of that message and stop when we get to C:\. It explains a lot.

This login manager isn't just broken, it goes right through broken and comes out on the other end.

## Saturday, April 10, 2010

### Can a website crash your RHEL5 desktop?

A few months ago I started having trouble with my RHEL5 desktop when running Firefox 3.5. On a few websites, the entire system would crash pretty consistently. It took a long time, but I finally managed to find the problem and get a fix in there.

My solution is documented on the YDN blog. Please leave comments there.

## Friday, April 09, 2010

### Analysis of the bandwidth and latency of YUIblog readers

A few months ago, Eric Miraglia from the YUI team helped me run some analysis on the types of network connections that were coming in to YUIBlog. In this article on the YUI blog, I've published the results of my analysis.

## Thursday, April 01, 2010

### Speaking at IPC Spring in Berlin

I'll be speaking at IPC Spring in Berlin between May 31 and June 2 this year. My talk is on MySQL partitioning. It will have more information than the talk I did at ConFoo in March. The talk will be on Tuesday, June 1st.

If you're in or around Berlin at that time, drop in.

## Monday, March 22, 2010

### InnoDB's tablespace ids and Partitions

There are times when what you have is a partially running database and a bunch of backup innodb tablespace files (the .ibd files). If you're using innodb_file_per_table, then you have a separate .ibd file for each InnoDB table.

Now, you have your running database with a bunch of tables, and you want to replace some of them with the backup .ibd files. According to the MySQL docs, you'd do this:
1. ALTER TABLE foo DISCARD TABLESPACE; (this deletes the current .ibd file)
2. copy the old .ibd file into your database directory
3. ALTER TABLE foo IMPORT TABLESPACE;
Assuming your .ibd file was from the same database and you did not drop the table and recreate it sometime between when you made the backup .ibd and now, this should work. Except... if you use partitions. If your table foo uses partitions, ie, its create statement was something like this:
CREATE TABLE foo (
...
) PARTITION BY ... (
PARTITION p0 ...,
);
In this case, you cannot discard the tablespace, and the first alter command throws an error:
mysql> ALTER TABLE foo DISCARD TABLESPACE;

ERROR 1031 (HY000): Table storage engine for 'foo' doesn't have this option

I have not investigated if there are workarounds for this, but I do have a little more information on what's happening. Remember that each .ibd file is a tablespace. For a partitioned table, there are multiple .ibd files, one for each partition. The table's files look like this:
foo.frm
foo.par
foo#P#p0.ibd
foo#P#p1.ibd
...

Where p0, p1, etc. are the partition names that you specified in the create statement. Each partition is a different tablespace and has its own tablespace id. When you create an InnoDB table without partitioning, the internal tablespace id counter is incremented by 1. When you create an InnoDB table with paritions, the internal tablespace id counter is incremented by the number of partitions. The actual tablespace id is stored in each partition's .ibd file somewhere within the first 100 bytes. I have not attempted to find out where exactly though.

## Friday, March 19, 2010

### WebDU

Wow, it's been another couple of weeks that I haven't updated this blog. Most of my time's gone in travelling. I was at ConFoo last week, and have a pending blog post about that, though it will be on the YDN blog. Next month I'll spend some time in India and then in May I head over to Bondi beach in Australia for WebDU.

I'll be speaking about web performance there and will probably do a pub event as well. Stay posted for more details. In any event, if Sydney is where you're at, then go register for WebDU. Geoff and crew put on a great show last year and it only looks to be getting better.

## Sunday, February 28, 2010

### Convert IP to Geo info using YQL

For my missing kids hack, I needed to convert an IP address to a US or Canadian 2 letter state code. This should have been pretty straightforward, but it turned out to require a little more effort than I initially wanted to put in.

First, the easy way. Rasmus Lerdorf has a web service that takes in an IP address and based on the MaxMind data, returns a bunch of information including the country and state/region code. I initially decided to use this. His example page is pretty self-explanatory, so I won't re-document it here. The problem is that this service was really slow and increased page load time a lot, so I scrapped the idea.

I then started looking through YQL. YQL has a whole bunch of geo stuff, but nothing that specifically turns an IP address into a WoEID or a country/state code. I then looked at the community supported tables and found the ip.location table that uses the ipinfodb.com wrapper around the MaxMind database. This returned everything I needed, but the only problem was that the state was returned as a string rather than a two character code. This is the query:
SELECT * From ip.location Where ip=@ip

The output looks like this:
{
"query":{
"count":"1",
"created":"2010-02-28T01:24:30Z",
"lang":"en-US",
"updated":"2010-02-28T01:24:30Z",
"uri":"http://query.yahooapis.com/v1/yql?q=select+*+from+ip.location+where+ip%3D%27209.117.47.253%27",
"results":{
"Response":{
"Ip":"209.117.47.253",
"Status":"OK",
"CountryCode":"US",
"CountryName":"United States",
"RegionCode":null,
"RegionName":null,
"City":null,
"ZipPostalCode":null,
"Latitude":"38",
"Longitude":"-97",
"Timezone":"-6",
"Gmtoffset":"-6",
"Dstoffset":"-5"
}
}
}
}

Now it's pretty trivial to build an array that maps from state name to state code, but I'd have to keep growing that as I added support for more countries, so I decided against that route. Instead I started looking at how I could use the geo APIs to turn this information into what I wanted. Among other things, the data returned also contained the latitude and longitude of the location that the IP was in. I decided to do a reverse geo map from the lat/lon to the geo information. The only problem is that the geo API itself doesn't do this for you.

Tom Croucher then told me that the flickr.places API could turn a latitude and longitude pair into a WoEID, so I decided to explore that. This is the query that does it:
SELECT place.woeid From flickr.places
Where lat=@lat And lon=@lon

Now I could tied the two queries together and get a single one that turns an IP address to a WoEID:
SELECT place.woeid From flickr.places
Where (lat, lon) IN
(
SELECT Latitude, Longitude From ip.location
Where ip=@ip
)

This is what the output looks like:
{
"query":{
"count":"1",
"created":"2010-02-28T01:25:34Z",
"lang":"en-US",
"updated":"2010-02-28T01:25:34Z",
"uri":"http://query.yahooapis.com/v1/yql?q=SELECT+place.woeid+From+flickr.places%0A+Where+%28lat%2C+lon%29+IN%0A+++%28%0A++++++SELECT+Latitude%2C+Longitude+From+ip.location%0A+++++++Where+ip%3D%27209.117.47.253%27%0A+++%29",
"results":{
"places":{
"place":{
"woeid":"12588378"
}
}
}
}
}

The last step of the puzzle was to turn this WoEID into a country and state code. This I already knew how to do:
SELECT country.code, admin1.code
From geo.places
Where woeid=@woeid

country.code gets us the two letter ISO3166 country code while admin1.code gets us a code for the local administrative region. For the US and Canada, this is simply the country code followed by a hyphen, followed by the two letter state code. Once I got this information, I could strip out the country code and the hyphen from admin1.code and get the two letter state code.

My final query looks like this:
SELECT country.code, admin1.code From geo.places
Where woeid IN
(
SELECT place.woeid From flickr.places
Where (lat, lon) IN
(
SELECT Latitude, Longitude From ip.location
Where ip=@ip
)
)

And the output is:
{
"query":{
"count":"1",
"created":"2010-02-28T01:26:32Z",
"lang":"en-US",
"updated":"2010-02-28T01:26:32Z",
"results":{
"place":{
"country":{
"code":"US"
},
"code":"US-KS"
}
}
}
}
}

Paste this code into the YQL console, make sure you've selected "Show community tables" and get the REST API from there. It's a terribly roundabout way to get something that should be a single API call, but at least from my application's point of view, I only need to call a single web service. Now if only we could convince the guys at missingkidsmap.com to use WoEIDs instead of state codes, that would make this all a lot easier.

Have I mentioned how much I like YQL?

## Saturday, February 27, 2010

### Closures and Function Currying

A few weeks ago, someone emailed me with this question:
Assume i have test.js file with code
function createAdder(x) {
return function(y) {
return x + y;
}
}

... My doubt: what's actually happening in the LINE 1-4?
I promised him an answer as a blog post, so here it is.

#### Currying

What we see here is something called function currying. It's common in mathematics and is named after one of its inventors — Haskell Curry. In short, currying converts a single function that takes in multiple arguments to multiple functions each of which takes in a single argument. This is particularly useful when caching the intermediate steps for later reuse with different parameters is of use. The classic example is that of operating on two numbers:
function divide(a) {
return function(b) {
return b/a;
}
}

var divide_2 = divide(2);
var divide_3 = divide(3);

In the above code, divide_2 is a function that takes in one argument and divides it by 2, returning the result. This is a fairly trivial and not very useful example because there are easier ways to divide a number by two. It becomes more useful though, when we need to do a bunch of expensive processing to get to each of the inner results. Consider this code instead:
function hash(salt) {
// do some expensive processing on salt
var hash1 = process(salt);
return function(data) {
// cheap processing of data with hash1
return data + hash1;
};
}

var sign1 = hash(salt1);   // sign1 is a function that signs data with salt1

var signature = sign1(some_data);

In the above code, the outer function does a bunch of expensive processing, and its result is stored in the hash1 variable. This variable is available to the inner function whenever it is called because of the closure that's created. When the inner function is called, it simply uses the value of hash1 without having to redo the processing. Now we could have called process() externally and cached its result, but then the hash1 would be exposed. This may not be something we want to do either because it needs to be abstracted out, or because its value is sensitive.

#### Closures

This all works because of closures. In short, a variable will continue to exist as long as code that can see it can be run. In the above cases, the inner functions are returned and their references stored in global variables. This makes the lifetime of these inner functions global, ie, they will exist as long as their new containing scope exists. These functions do not, however, get the new scope, so the variables they can see are exactly what they could see when they were defined. In the divide example, the inner function sees the variable a, therefore a will exist for as long as the inner function exists. When we create divide_2, the value of a is set to 2, and this is what the inner function (which is now stored in divide_2) sees. When we create divide_3, a new a is created, this time with value 3, and a new inner function is created (which is now stored in divide_3) and this function sees the new value of a. This is a completely new execution scope than when divide(2) was called. So getting back to the example my friend asked about, this is what happens:
1. createAdder(2): At this point, the argument x is set to the value 2, and the inner function is returned and stored in add2. This function remembers the value of x and uses it when it has to be called.
2. createAdder(5): At this point, the argument x is set to the value 5. Note that this is a new invocation of createAdder and does not share the same memory as the first invocation, so these are two completely different variables named x, both living in different scopes.
3. add2(10): At this point, the first inner function is called with the argument 10, which is stored in y. This function remembers the value of x as 2 and computes 2 + 10 and returns its value
4. add5(10): The second instance of the inner function is called with the argument 10, which is stored in y. This function remembers the value of x as 5 from when it was called, and computes 5 + 10 and returns its value
Now this whole explanation of closures would not be complete without one more subtle note that most people tend to forget about. The inner functions see the variables that were defined within its containing scope regardless of where in that containing scope they were defined or when their values were set. This means that if you change the value of a variable after defining an inner function, then the inner function will see the new value. Here's an example:
function foo(x) {
var g=function(y) {
return x+y;
};
x=x*x;
return g;
}

Notice that the value of x was changed after the inner function g was defined, yet g sees the new value of x. This is particularly important when you use a loop control variable inside a closure. The closure will see the last value of the loop control variable and not a different value on each iteration.

Update: 2010-03-04 t3rmin4t0r has a much more useful example of currying on his blog.

## Friday, February 19, 2010

### Missing kids on your 404 page

It's been a long time since I last posted, and unfortunately I've been unable to churn out a post every week. The month of February has been filled with travel, so I haven't had much time to write.

My report on FOSDEM is up on the YDN blog, so I haven't been completely dormant. I also did some stuff at our internal hack day last week. This post is about one of my hacks.

The idea is quite simple. People land up on 404 pages all the time. 404 pages are pages that have either gone missing, or were never there to begin with. 404 is the HTTP error code for a missing resource. Most 404 pages are quite bland, simply stating that the requested resource was not found, and that's it. Back when I worked at NCST, I changed the default 404 page to use a local site search based on the requested URL. I used the namazu search engine since I was working on it at the time.

This time I decided to do something different. Instead of searching the local site for a missing resource, why not engage the user in trying to find missing kids.

I started with trying to find an API for missingkids.com and ended up finding missingkidsmap.com. This service takes the data from Missing Kids and puts it on a google map. The cool thing about the service was that it could return data as XML.

Looking through the source code, I found the data URL:
http://www.missingkidsmap.com/read.php?state=CA

The state code is a two letter code for states in the US and Canada. To get all kids, just pass in ZZ as the state code.

The data returned looks like this:
<locations>
<maplocation zoom="5"
state_long="-119.838867"
state_lat="37.370157"/>
<location id="1"
firstname="Anastasia"
lastname=" Shearer "
picture="img width=160 target=_new src=http://www.missingkids.com/photographs/NCMC1140669c1.jpg"
picture2="img width=160 target=_new src=http://www.missingkids.com/photographs/NCMC1140669e1.jpg"
medpic = "img width=60 border=0 target=_new src=http://www.missingkids.com/photographs/NCMC1140669c1.jpg"
smallpic="img width=30 border=0 target=_new src=http://www.missingkids.com/photographs/NCMC1140669c1.jpg"
policenum="1-661-861-3110"
policenum2=""
st=" CA"
city="BAKERSFIELD"
missing="12/26/2009"
status="Endangered Runaway"
age="16"
url="1140669"
lat="35.3733333333333"
lng="-119.017777777778"/>
...
</locations>

Now I could keep hitting this URL for every 404, but I didn't want to kill their servers, so I decided to pass the URL through YQL and let them cache the data. Of course, now that I was passing it through YQL, I could also do some data transformation and get it out as JSON instead of XML. I ended up with this YQL statement:
SELECT * From xml

Pass that through the YQL console to get the URL you should use. The JSON I got back looked like this:
{
"query":{
"count":"1",
"created":"2010-02-19T07:30:44Z",
"lang":"en-US",
"updated":"2010-02-19T07:30:44Z",
"results":{
"locations":{
"maplocation":{
"state_lat":"40.313043",
"state_long":"-94.130859",
"zoom":"4"
},
"location":[{
"age":"7",
"city":"OMAHA",
"firstname":"Christopher",
"id":"Szczepanik",
"lastname":"Szczepanik",
"lat":"41.2586111111111",
"lng":"-95.9375",
"medpic":"img width=60 border=0 target=_new src=http://www.missingkids.com/photographs/NCMC1141175c1.jpg",
"missing":"12/14/2009",
"picture":"img width=160 target=_new src=http://www.missingkids.com/photographs/NCMC1141175c1.jpg",
"picture2":"",
"policenum":"1-402-444-5600",
"policenum2":"",
"smallpic":"img width=30 border=0 target=_new src=http://www.missingkids.com/photographs/NCMC1141175c1.jpg",
"st":" NE",
"status":"Missing",
"url":"1141175"
},
...
]
}
}
}
}

Step 2 was to figure out whether the visitor was from the US and Canada, and if so, figure out which state they were from and pass that state code to the URL.

This is fairly easy to do at Yahoo!. Not so much on the outside, so I'm going to leave it to you to figure it out (and please let me know when you do).

In any case, my code looked like this:
$json = http_get($missing_kids_url);
$o = json_decode($json, 1);
$children =$o['query']['results']['locations']['location'];

$child = array_rand($children);

print_404($child);  http_get is a function I wrote that wraps around curl_multi to fetch and cache locally a URL. print_404 is the function that prints out the HTML for the 404 page using the $child data object. The object's structure is the same as each of the location elements in the JSON above. The important parts of print_404 are:
function print_404($child) {$img = preg_replace('/.*src=(.*)/', '$1',$child["medpic"]);
$name =$child["firstname"] . " " . $child["lastname"];$age = $child['age'];$since = strtotime(preg_replace('|(\d\d)/(\d\d)/(\d\d\d\d)|', '$3-$1-$2',$child['missing']));
if($age == 0) {$age = ceil((time()-$since)/60/60/24/30);$age .= ' month';
}
else
$age .= ' year';$city = $child['city'];$state = $child['st'];$status = $child['status'];$police = $child['policeadd'] . " at " .$child['policenum'];

?>
<html>
...
<p>
<strong>Sorry, the page you're trying to find is missing.</strong>
</p>
<p>
We may not be able to find the page, but perhaps you could help find this missing child:
</p>
<div style="text-align:center;">
<img style="width:320px; padding: 1em;" alt="<?php echo $name ?>" src="<?php echo$img ?>"><br>
<div style="text-align: left;">
<?php echo $age ?> old <?php echo$name ?>, from <?php echo "$city,$state" ?> missing since <?php echo strftime("%B %e, %Y", $since); ?>.<br> <strong>Status:</strong> <?php echo$status ?>.<br>
<strong>If found, please contact</strong> <?php echo $police ?><br> </div> </div> ... </body> </html> <?php }  Add in your own CSS and page header, and you've got missing kids on your 404 page. The last thing to do is to tell apache to use this script as your 404 handler. To do that, put the page (I call it 404.php) into your document root, and put this into your apache config (or in a .htaccess file): ErrorDocument 404 /404.php  Restart apache and you're done. Update: 2010-02-24 To see it in action, visit a missing page on my website. eg: http://bluesmoon.info/foobar. Update 2: The code is now on github: http://github.com/bluesmoon/404kids Update: 2010-02-25 Scott Hanselman has a Javascript implementation on his blog. Update: 2010-03-28 There's now a drupal module for this. ## Friday, January 29, 2010 ### Pretty print cron emails We have a lot of scripts that run through cron and the output gets emailed by cron to whoever owns the cron job, or to a mailing list or something. Unfortunately, the headers for these mails are ugly and could use a lot of work. The primary problem I had with these mails was that the subject line could not be customised. Other smaller problems were that the From: address and Return-Path didn't have good values. A sample header looks like this: From: Cron Daemon <root@hostname.domain> To: someone@somewhere.com Subject: Cron <root@hostname> /usr/local/bin/foobar --whatever  Of the above, only the To: address makes any sense. The first thing I tried to change was the subject line. This is fairly easy using the mail program: 10 15 * * * /usr/local/bin/foobar --whatever | mail -s "Pretty subject" someone@somewhere.com  The mail now looks like this: From: Cron Daemon <root@hostname.domain> To: someone@somewhere.com Subject: Pretty subject  This is a big step forward, because we can now filter emails based on subject. It still doesn't let us change the from address though. That's when I remembered using sendmail a long time ago to send mail through perl scripts, so I had a look through the sendmail man page, and found a few interesting flags: -F full_name Set the sender full name. This is used only with messages that have no From: message header. -f sender Set the envelope sender address. This is the address where delivery problems are sent to, unless the message contains an Errors-To: message header.  This lets you set the From header, but there's no way to change the Subject. Since we can't use mail and sendmail together, we can either set the From address or the Subject, but not both. I then remembered the -t flag to sendmail that tells it to read the header from the message itself. The only thing left to do was to add the header into the output of every command. Since I couldn't easily do this in cron, I wrote my own script to do it. It's called pmail. The p is for pretty. Get the script now The meat of the script is this one line: ( echo -e "From:$from_name <$from>\r\nSubject:$subject\r\nTo: $to\r\n\r\n"; cat ) | /usr/sbin/sendmail -t  It writes the From and Subject headers, then leaves a blank line, and then passes its STDIN directly on to sendmail. The script has 3 optional parameters — the sender's address, sender's name and subject, and one mandatory parameter — the recipient's address(es). To use it, set your cron line something like this: * * * * * /usr/local/bin/foobar --whatever | \ pmail -F "Philip Tellis" -f "philip@bluesmoon.info" -s "Pretty mail from foobar" someone@somewhere.com  Note: lines wrapped for readability. You can also add the current date to the subject line like this: ... -s "Foobar mail on$( date +\%Y-\%m-\%d )" ...

Remember that you have to escape % signs in crontab lines otherwise cron will translate them to newline characters.

Umm, you also need to have sendmail set up correctly, and doing that is your problem :)

It's not complete yet, but it works for my use case. If you like it, use it. It's under the BSD license. I'll probably add a Reply-To option if it makes sense for me, and at some point add in data validation too, but not today.

Enjoy.

## Tuesday, January 26, 2010

### Speaking at FOSDEM 2010

I'll be speaking at FOSDEM 2010 in Brussels, Belgium on the 6th and 7th of February. I'll be speaking about the YUI Flot library. Registration is free, so if you're in the area, just show up. If anyone's interested in a performance BoF, let's do that as well.

## Monday, January 25, 2010

### Speaking at ConFoo

I'll be speaking at ConFoo in Montreal this March. I have two talks on performance and scaling. If you're in the area, drop in. The conference line-up is great, and if PHP Quebec last year was any indication, it should be very enlightening.

## Tuesday, January 19, 2010

### Bandwidth test v1.2

After testing this out for about a week, I'm ready to release v1.2 of the bandwidth testing code.

#### Changes

The changes in this release are all statistical, and related to the data I've collected while running the test.
1. Switch from the geometric mean to the arithmetic mean, because data for a single user is more or less centrally clustered.

This is not true across users, but for readings for a single user (single connection actually), I've found that the data does not really have outliers.

This allows us to download fewer bytes, and get more consistent results across runs.
3. Add random sampling. In previous versions, the test would be run for every page view. By adding random sampling, we now only run it on a percentage of page views.

You can now set PERFORMANCE.BWTest.sample to a number between 0 and 100, and that's the percentage of page views that will be tested for bandwidth. Note that this is not an integer, so you can set it to 0.1 or 10.25, or anything as long as it's between 0 and 100 (both inclusive). Setting it to 0 means that the test isn't run for anyone, so you probably don't want that, and 100 means that the test is run for everybody. This is also the default.
4. Fire an onload event when the script completes loading. This is mainly so that you can asynchronously load the script so that it does not affect your page load time. Instead of checking for script.onload or onreadystatechange, you can just implement the PERFORMANCE.BWTest.onload method.
You can see the entire history on github.

...===...


text:
## So Close … and yet SO Far – Mark 12:28-34

Sermon: So Close … and yet SO Far.

Text: Mark 12:28-34

Sermon #52 in the Series: Mark ~ The Christ, The Kingdom and The Cross


text:
Subject:
Mathematics, Algebra
Material Type:
Module
Level:
Community College / Lower Division, College / Upper Division
Provider:
Ohio Open Ed Collaborative
Tags:
Omt0192
Language:
English
Media Formats:
Text/HTML

VSP-0050: Abstract Vector Spaces

Overview

We state the definition of an abstract vector space, and learn how to determine if a given set with two operations is a vector space. We define a subspace of a vector space and state the subspace test. We find linear combinations and span of elements of a vector space.

https://ximera.osu.edu/la/LinearAlgebra/VSP-M-0050/main


text:
# Computer Modern Bold SF has rounded corners

When experimenting with the Beamer class I noticed that when using the default SF font (which is Computer Modern AFAIK) in boldface the corners are rounded. It looks like completely different font. (And I think it is. Look at the shape of the 'o' (oh) which became squared instead of rounded.) I have never seen this before on fonts: bold is usually a bold version of the original font. Is this normal/intentional? Why?

I am creating a template for a presentation in company style so I am not allowed to change the font.

Compare Helvetica in normal and boldface, and compare Computer Modern Sans in normal and boldface.

\documentclass{article}
\renewcommand{\familydefault}{\sfdefault}
\usepackage{helvet}
\begin{document}
Helvetica\par
\textbf{Helvetica}
\end{document}

\documentclass{article}
\renewcommand{\familydefault}{\sfdefault}
%\usepackage{helvet}
\begin{document}
Computer Modern\par
\textbf {Computer Modern}
\end{document}

On a side note, for this post I had to create two documents. For the life of it, I have not been able to change the font back to Computer Modern Sans. Although I set the document default to Sans, this command would show Computer Modern with serif. Only when I comment out \usepackage{helvet} I can get the Computer Modern font.

\documentclass{article}
\renewcommand{\familydefault}{\sfdefault}
\usepackage{helvet}
\begin{document}
Helvetica\par
\fontfamily{cmr}\selectfont
Expected Computer Modern Sans here
\end{document}

• The a of Helvetica is quite different as well though! – albert Apr 28 '18 at 16:16
• you need cmss instead of cmr for the sans font. – Ulrike Fischer Apr 28 '18 at 16:17
• @Albert: yes, you are right. I don't deny that but it is by far not that obvious as with Computer Modern. – jlinkels Apr 28 '18 at 20:05
• @Ulrike Fischer: Makes sense! cmSS instead or cmR. Really impossible to find in Google. Always ending up at the same guides when searching for "setting font to Computer Modern" – jlinkels Apr 28 '18 at 20:12

Font design is the choice of the font designer.

But regarding the font change: You need to use cmss instead of cmr if you want the computer modern sans font:

\documentclass{article}
\renewcommand{\familydefault}{\sfdefault}
\usepackage{helvet}
\begin{document}
Helvetica\par
\fontfamily{cmss}\selectfont
Expected Computer Modern Sans here
\end{document}


text:
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Introduction

This is the final and concluding part of my series on ‘Practical Machine Learning with R and Python’. In this series I included the implementations of the most common Machine Learning algorithms in R and Python. The algorithms implemented were

1. Practical Machine Learning with R and Python – Part 1 In this initial post, I touch upon regression of a continuous target variable. Specifically I touch upon Univariate, Multivariate, Polynomial regression and KNN regression in both R and Python
2. Practical Machine Learning with R and Python – Part 2 In this post, I discuss Logistic Regression, KNN classification and Cross Validation error for both LOOCV and K-Fold in both R and Python
3. Practical Machine Learning with R and Python – Part 3 This 3rd part included feature selection in Machine Learning. Specifically I touch best fit, forward fit, backward fit, ridge(L2 regularization) & lasso (L1 regularization). The post includes equivalent code in R and Python.
4. Practical Machine Learning with R and Python – Part 4 In this part I discussed SVMs, Decision Trees, Validation, Precision-Recall, AUC and ROC curves
5. Practical Machine Learning with R and Python – Part 5  In this penultimate part, I touch upon B-splines, natural splines, smoothing spline, Generalized Additive Models(GAMs), Decision Trees, Random Forests and Gradient Boosted Treess.

In this last part I cover Unsupervised Learning. Specifically I cover the implementations of Principal Component Analysis (PCA). K-Means and Heirarchical Clustering. You can download this R Markdown file from Github at MachineLearning-RandPython-Part6

1.1a Principal Component Analysis (PCA) – R code

Principal Component Analysis is used to reduce the dimensionality of the input. In the code below 8 x 8 pixel of handwritten digits is reduced into its principal components. Then a scatter plot of the first 2 principal components give a very good visial representation of the data

library(dplyr)
library(ggplot2)
#Note: This example is adapted from an the example in the book Python Datascience handbook by
# Jake VanderPlas (https://jakevdp.github.io/PythonDataScienceHandbook/05.09-principal-component-analysis.html)

# Read the digits data (From sklearn datasets)
# Create a digits classes target variable
digitClasses <- factor(digits$X0.000000000000000000e.00.29) #Invoke the Principal Componsent analysis on columns 1-64 digitsPCA=prcomp(digits[,1:64]) # Create a dataframe of PCA df <- data.frame(digitsPCA$x)
# Bind the digit classes
df1 <- cbind(df,digitClasses)
# Plot only the first 2 Principal components as a scatter plot. This plot uses only the
# first 2 principal components
ggplot(df1,aes(x=PC1,y=PC2,col=digitClasses)) + geom_point() +
ggtitle("Top 2 Principal Components")

1.1 b Variance explained vs no principal components – R code

In the code below the variance explained vs the number of principal components is plotted. It can be seen that with 20 Principal components almost 90% of the variance is explained by this reduced dimensional model.

# Read the digits data (from sklearn datasets)
# Digits target
digitClasses <- factor(digits$X0.000000000000000000e.00.29) digitsPCA=prcomp(digits[,1:64]) # Get the Standard Deviation sd=digitsPCA$sdev
# Compute the variance
digitsVar=digitsPCA$sdev^2 #Compute the percent variance explained percentVarExp=digitsVar/sum(digitsVar) # Plot the percent variance exlained as a function of the number of principal components #plot(cumsum(percentVarExp), xlab="Principal Component", # ylab="Cumulative Proportion of Variance Explained", # main="Principal Components vs % Variance explained",ylim=c(0,1),type='l',lwd=2, # col="blue") 1.1c Principal Component Analysis (PCA) – Python code import numpy as np from sklearn.decomposition import PCA from sklearn import decomposition from sklearn import datasets import matplotlib.pyplot as plt from sklearn.datasets import load_digits # Load the digits data digits = load_digits() # Select only the first 2 principal components pca = PCA(2) # project from 64 to 2 dimensions #Compute the first 2 PCA projected = pca.fit_transform(digits.data) # Plot a scatter plot of the first 2 principal components plt.scatter(projected[:, 0], projected[:, 1], c=digits.target, edgecolor='none', alpha=0.5, cmap=plt.cm.get_cmap('spectral', 10)) plt.xlabel('PCA 1') plt.ylabel('PCA 2') plt.colorbar(); plt.title("Top 2 Principal Components") plt.savefig('fig1.png', bbox_inches='tight') 1.1 b Variance vs no principal components – Python code import numpy as np from sklearn.decomposition import PCA from sklearn import decomposition from sklearn import datasets import matplotlib.pyplot as plt from sklearn.datasets import load_digits digits = load_digits() # Select all 64 principal components pca = PCA(64) # project from 64 to 2 dimensions projected = pca.fit_transform(digits.data) # Obtain the explained variance for each principal component varianceExp= pca.explained_variance_ratio_ # Compute the total sum of variance totVarExp=np.cumsum(np.round(pca.explained_variance_ratio_, decimals=4)*100) # Plot the variance explained as a function of the number of principal components plt.plot(totVarExp) plt.xlabel('No of principal components') plt.ylabel('% variance explained') plt.title('No of Principal Components vs Total Variance explained') plt.savefig('fig2.png', bbox_inches='tight') 1.2a K-Means – R code In the code first the scatter plot of the first 2 Principal Components of the handwritten digits is plotted as a scatter plot. Over this plot 10 centroids of the 10 different clusters corresponding the 10 diferent digits is plotted over the original scatter plot. library(ggplot2) # Read the digits data digits= read.csv("digits.csv") # Create digit classes target variable digitClasses <- factor(digits$X0.000000000000000000e.00.29)

# Compute the Principal COmponents
digitsPCA=prcomp(digits[,1:64])

# Create a data frame of Principal components and the digit classes
df <- data.frame(digitsPCA$x) df1 <- cbind(df,digitClasses) # Pick only the first 2 principal components a<- df[,1:2] # Compute K Means of 10 clusters and allow for 1000 iterations k<-kmeans(a,10,1000) # Create a dataframe of the centroids of the clusters df2<-data.frame(k$centers)

#Plot the first 2 principal components with the K Means centroids
ggplot(df1,aes(x=PC1,y=PC2,col=digitClasses)) + geom_point() +
geom_point(data=df2,aes(x=PC1,y=PC2),col="black",size = 4) +
ggtitle("Top 2 Principal Components with KMeans clustering")

1.2b K-Means – Python code

The centroids of the 10 different handwritten digits is plotted over the scatter plot of the first 2 principal components.

import numpy as np
from sklearn.decomposition import PCA
from sklearn import decomposition
from sklearn import datasets
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans

# Select only the 1st 2 principal components
pca = PCA(2)  # project from 64 to 2 dimensions
projected = pca.fit_transform(digits.data)

# Create 10 different clusters
kmeans = KMeans(n_clusters=10)

# Compute  the clusters
kmeans.fit(projected)
y_kmeans = kmeans.predict(projected)
# Get the cluster centroids
centers = kmeans.cluster_centers_
centers

#Create a scatter plot of the first 2 principal components
plt.scatter(projected[:, 0], projected[:, 1],
c=digits.target, edgecolor='none', alpha=0.5,
cmap=plt.cm.get_cmap('spectral', 10))
plt.xlabel('PCA 1')
plt.ylabel('PCA 2')
plt.colorbar();
# Overlay the centroids on the scatter plot
plt.scatter(centers[:, 0], centers[:, 1], c='darkblue', s=100)
plt.savefig('fig3.png', bbox_inches='tight')

1.3a Heirarchical clusters – R code

Herirachical clusters is another type of unsupervised learning. It successively joins the closest pair of objects (points or clusters) in succession based on some ‘distance’ metric. In this type of clustering we do not have choose the number of centroids. We can cut the created dendrogram mat an appropriate height to get a desired and reasonable number of clusters These are the following ‘distance’ metrics used while combining successive objects

• Ward
• Complete
• Single
• Average
• Centroid
# Read the IRIS dataset
iris <- datasets::iris
iris2 <- iris[,-5]
species <- iris[,5]

#Compute the distance matrix
d_iris <- dist(iris2)

# Use the 'average' method to for the clsuters
hc_iris <- hclust(d_iris, method = "average")

# Plot the clusters
plot(hc_iris)

# Cut tree into 3 groups
sub_grp <- cutree(hc_iris, k = 3)

# Number of members in each cluster
table(sub_grp)
## sub_grp
##  1  2  3
## 50 64 36
# Draw rectangles around the clusters
rect.hclust(hc_iris, k = 3, border = 2:5)

1.3a Heirarchical clusters – Python code

from sklearn.datasets import load_iris
import matplotlib.pyplot as plt
# Load the IRIS data set

# Generate the linkage matrix using the average method

#Plot the dendrogram
#dendrogram(Z)
#plt.xlabel('Data')
#plt.ylabel('Distance')
#plt.suptitle('Samples clustering', fontweight='bold', fontsize=14);
#plt.savefig('fig4.png', bbox_inches='tight')

Conclusion

This is the last and concluding part of my series on Practical Machine Learning with R and Python. These parallel implementations of R and Python can be used as a quick reference while working on a large project. A person who is adept in one of the languages R or Python, can quickly absorb code in the other language.

Hope you find this series useful!

More interesting things to come. Watch this space!

References

1. Statistical Learning, Prof Trevor Hastie & Prof Robert Tibesherani, Online Stanford
2. Applied Machine Learning in Python Prof Kevyn-Collin Thomson, University Of Michigan, Coursera

To see all posts see ‘Index of posts


text:
# Uniform Distribution Formula

## Theoretical Mean Formula

 $\mu$ = $\frac{x+y}{2}$

## Standard Deviation Formula

 $\sigma = \sqrt{ \frac{(y-x)^{2}}{12}}$

## Example of Uniform Distribution

Example: The data in the table below are 55 times a baby yawns, in seconds, of a 9-week-old baby girl.

 10.4 19.6 18.8 13.9 17.8 16.8 21.6 17.9 12.5 11.1 4.9 12.8 14 22.8 20.8 15.9 16.3 13.4 17.1 14.5 19 22.8 1.3 0.7 8.9 11.9 10.9 7.3 5.9 3.7 17.9 19.2 9.8 5.8 6.9 2.6 5.8 21.7 11.8 3.4 2.1 4.5 6.3 10.7 8.9 9.7 9.1 7.7 10.1 3.5 6.9 7.8 11.6 13.8 18.6
• The sample mean = 11.49
• The sample standard deviation = 6.23.

As assumed, the yawn times in secs, it follows a uniform distribution between 0 to 23 seconds(Inclusive).

So, it is equally likely that any yawning time is from 0 to 23.

In this example,

The theoretical mean = $\mu$ = $\frac{x+y}{2}$ $\mu$ = $\frac{0+23}{2}$ = 11.50

standard deviation = $\sigma = \sqrt{ \frac{(y-x)^{2}}{12}}$

standard deviation = $\sqrt{\frac{(23-0)2}{12}}$ = 6.64 seconds.


text:
# Tag Info

5

The state of Equation * is local unitary equivalent to all of the four Bell states. That is, there exists a unitary matrix of tensor-product structure $V_a \otimes V_b$, such that $U_{ab}|+\rangle\otimes |+\rangle = V_a \otimes V_b \frac{1}{\sqrt{2}}\left(|00\rangle + |11\rangle\right)$ In terms of its entanglement properties, Equation * is completely ...

4

To obtain the stabilisers of a graph state, from its adjacency matrix: Change all 1s to Zs Change all 0s to identity operators Put X operators on the diagonal Each row then represents a stabiliser of the graph state, and any nontrivial stabiliser is a product of one or more rows.

4

Felix Huber's answer is correct, but I thought I'd fill in a few more details. You've written the two-qubit state of ($\star$) as $(|0\rangle|+\rangle+|1\rangle|-\rangle)/\sqrt{2}$. Notice that if you apply a Hadamard to the second qubit, you change it to $(|0\rangle|0\rangle+|1\rangle|1\rangle)/\sqrt{2}$, a Bell state. The point is that from the ...

4

The number of Bell pairs required to construct a given graph state can easily be given an upper bound: $|V|-1$, where $V$ is the set of vertices. You do this simply by preparing the entire state at one site, and teleporting all the other qubits to the relevant party. I wonder if this is actually all there is to it? If we assume that the entire graph is ...

2

I'm not familiar with how graph states extend to qudits, so let me just answer for the specific case of qubits. Consider a graph $G$, and we create the corresponding graph state $|G\rangle$ by placing a qubit on every vertex in the $|+\rangle$ state, and applying a controlled-phase gate along every edge. Now, take a bipartition of $G$. On either side of ...

2

the measurement is carried out on each individual qubit and the measurement on one qubit will not change the state of another qubit This is an incorrect statement. If the state that you are measuring is entangled (which it very much is for the 2D cluster state), measuring the state of one qubit absolutely changes the state of another qubit. The trivial ...

2

It is incorrect to use modulo arithmetic in this context. Instead finite field arithmetic should be applied. In $\textrm{GF}(4) = \{0, 1, x, x^2\}$ where $x^2 = x + 1$ and conjugation of $a$ is defined as $\bar{a} = a^2$. Addition, multiplication and conjugation tables are then as follows: In this picture we have $0 \equiv 0$, $1 \equiv 1$, $2 \equiv x$, ...

2

Imgine that there is a bit string $\vec{k}\in\{0,1\}^n$. We use this to specify sites (bit value 1) where an error has occurred, and sites (bit value 0) where no error has occurred. The number of 1s in the bit string is $k$. The probability of this particular error arising is then $p^k(1-p)^{n-k}$ because there are $k$ sites with an error and $n-k$ sites ...

2

If you start with one graph state, which is an eigenstate of stabilizer (each of which comprises an X tensored with a bunch of Zs), then the other eigenstates of those stabilizers are the original state acted on by Z rotations (take all possible combinations). To see this note the commutation and anticommutation relations between a tensor product of Zs and ...

2

The calculation of the resulting state of the described circuit: After applying the same circuit identities described in this answer (and here) to the connected question we will obtain a "simplified" circuit: Here are the links to the initial, first intermediate, second intermediate and final circuits presented in the Quirk. Whenever it was ...

2

Because $CNOT = I\otimes H \cdot CZ \cdot I\otimes H$ as was mentioned here, and because $CZ(q_1, q_2) = CZ(q_2, q_1)$, we can rewrite the circuit in this way (by adding Hadamards as needed): The link to the circuit created with Quirk. As one can see from the circuit above, after the first Hadamard gate and two $CNOT$ gates, we will have a GHZ state for the ...

2

We need to go throught the gates one by one to understand what's happening. We have to keep a few things in mind. $H|0\rangle=\frac{1}{\sqrt2}(|0\rangle + |1\rangle) = |+\rangle$ and $H|1\rangle=\frac{1}{\sqrt2}(|0\rangle - |1\rangle) = |-\rangle$. In the Hadamard basis the $Z$ gate acts like the $X$ gate. Namely $Z|+\rangle=|-\rangle$ and $Z|-\rangle=|+\... 2 You can think of a cluster state as a graph state, where the graph's vertices are on some$d$-dimensional lattice (normally just$2$-dimensional). Each vertex represents a qubit in the$|+\rangle$state, and two vertices that are next to each other on the lattice may be connected, which means that a$CZ$or controlled-$Z$gate has been applied to them. ... 1 About checking. I do not quite understand why you can't check how the GHZ-state was teleported in the most ordinary ways: in addition to measuring the state immediately after teleportation, as well as by YYX, YXY, XYY, XXX measurements, you can inverse your GHZ-state and make sure that all 0 are obtained, e.g. like as for this with the most usual GHZ-state ... 1 I think it is worth trying to understand the circuit that the authors really want to implement: Here they produce the Bell state they want to teleport onto the final two qubits, and cluster state (personally, I wouldn't call it a cluster state after they've added the two extra hadamards). Then they do two single-qubit teleportation protocols to make the ... 1 I'm not sure I understand the question, since this seems quite straightforward. Graph states are Clifford states, so for a state on$n$qubits, the set of stabilizers has$n$generators looking like $$i^k P_1\otimes\ldots\otimes P_n$$ where$k\in\{0,1,2,3\}$and the$P$'s are Pauli$X,Y,$or$Z$operators. (For graph states, these stabilizers look like$\...

Only top voted, non community-wiki answers of a minimum length are eligible