text: Tip: Spindle centerline is the same height as the center of the kingpin boss for calculating ride height. What is Kingpin Inclination King pin inclination (also referred to as the steering axis inclination, SAI or KPI) is quite simply the axis about which the wheels steer. c. King Pin axis inclination (a.k.a. In physics, if an object is pushed down a slope, the slope’s angle of inclination affects the object’s acceleration. The effect of kingpin inclination angle and wheel offset on various vehicle performance metrics such as steering effort, vehicle handling, and steering system vibration is described in this paper. This angle is designed to give a certain kingpin inclination relative to camber. effect of Kingpin Inclination Angle (KPIA) and wheel offset on vehicle performance metrics such as steering effort, vehicle handling and steering system vibration. Then, run the 1/4-20 tap through the setscrew holes, and ream or hone the bore to 11/16". This gives an unfamiliar feel to the steering as is the reason that it is used in harmony with KPI. It is a directional control angle with fixed relationship to camber settings. This angle is designed to give a certain kingpin inclination relative to camber. Viewed 517 times 1. The invention discloses a system and method for measuring a steering angle of a vehicle. Method and apparatus are disclosed for determining the caster angle and the steering angle inclination of a steerable wheel on a vehicle. The inward tilt of the kingpin at the top is known as kingpin inclination (KPI). 12-3) is measured in degrees from the center line of the ball joint or kingpin to true vertical (0). (8). Budget to Premium Coilovers – Whats The Real Difference? It can be estimated from the known values of height and distance of the object. 000. 51 Therefore, with symmetric geometry, the effect of right steer will roll the car to the left causing a diagonal weight shift. Information and translations of angle of inclination or dip in the most comprehensive dictionary definitions resource on the web. This is called the angle of inclination of a straight line.. We notice that if the gradient changes, then the value of $$\theta$$ also changes, therefore the angle of inclination of a line is related to its gradient. • Caster: The angle between the vertical line and kingpin center On current suspension systems, the kingpin is set at an angle to the vertical plane when viewed from the front or rear of the vehicle. If there is no caster present this effect is symmetrical from side to side. But it was noted that a marked increase in this angle created negative effects, especially with the tyres at low pressure, the specific need arose, to reduce the camber angle almost to zero, this was also to achieve regular wear on the tyre, the problem was resolved by inclining the kingpin towards the lower part of the wheel. Definition of angle of inclination or dip in the Definitions.net dictionary. Castor does the same thing. For example, when you push a sled down an ice-covered hill, you can calculate the hill’s angle of inclination given the sled’s mass and acceleration. Toe Angle carries the same sign as Toe Distance. This angle is known as the king pin inclination. All cars have different amounts of KPI which has been designed in to work with the whole geometry system (Scrub, Caster, etc), Your email address will not be published. The purpose of the KPI is to produce vertical displacement of the vehicle in during steering in an upward direction. and calculating the SAI angle using the measured displacement, the steering angle, the steering axis inclination offset, the axle height above said plane and the camber angle of the wheel. It is a diagnostic angle. Well slope m equals the tangent of the angle of inclination so in … Scrub radius is the distance from the steer axis to the centre of the tyre contact patch at ground level and wheel centre offset is the distance from the kingpin axis to the wheel centre. The kingpin inclination is the angle, measured in degrees, that forms the line passing through the kingpin and the perpendicular to the ground, looking at the vehicle from the front. 4.3 Inclination of a line (EMBGD). A larger angle is usually given- 6-7° for this inclination - in order that the wheel may be ‘swiveled’ about a point on the road surface. This lifting effect produces a self centering torque similar to that of caster. Positive camber is defined as the tops of the wheels tipping away from the vehicle. Calculate the following geometric parameters: (i) Caster (ii) Mechanical Trail (iii) Kingpin Inclination Angle (iv) Kingpin Longitudinal Offset (v) Scrub Radius (vi) Kingpin Lateral Offset or Spindle Length GIVEN DATA: SOLUTION: Sign Convention: Geometric Representation: Caster (theta)… Your email address will not be published. This natural effect, which is vitally important, due to the inclination of the kingpin, derives from the fact that the wheel, when turning about this oblique axis, forms an inverted cone. Here's a sketch which draws a right triangle that is our geometric object of interest: So you want an angle in a right triangle and you know the lengths of the opposite side and the hypotenuse of the triangle. Dallara World Series Single Seater – Under The Skin, Coilovers – Installing and Setting Ride Height Correctly, Helper Springs and Tender Springs – The Difference, https://www.youtube.com/watch?v=_Ksbfn40adA. There is no specific formula to calculate KPI. Figure 8. We fit a complete range of tyre brands across all types of car from high performance and super cars to prestige models and family cars. If θ is the angle of inclination of a straight line l, then tanθ is called the slope of gradient of the line is denoted by "m". Caster angle, similar to KPI, causes the wheel to rise and fall as well. However i still need to work on my spoken english. The larger the KPI, the larger the effect. I'm having a hard time seeing why you'd want any at all. Consider a king pin axis X-Y. […] KPI (fig. Castor or Caster angle . After sensitivity analysis, the The angle between the kingpin axis and the wheel axis is equal to the algebraic sum of the kingpin inclination angle and the wheel camber angle and is defined as the included angle. we use a 10 degree kingpin angle and the initial negative camber setting at the right front is 3 degrees. Consult your chassis builder for proper caster angle. Let them tell you! The wheel is supported on a member which has freedom of motion along the fore-and-aft and transverse axes of the vehicle and freedom of rotation about a mutually perpendicular rotation axis. NaCoMM-2007-106. NaCoMM-2007-106. How to calculate angle of inclination attained by a weigh balance on unequal loading? Notify me of follow-up comments by email. To master the basic characteristics of steady-state cornering for a semitrailer, this paper summarises the current modelling methods for handling and stability and discusses their limitations. The invention as defined in claim 4 wherein said calculating step is performed in accordance with the following functional relationship: ##EQU28## where: SAI=steering axis inclination angle, Swivel pin or kingpin inclination is the lateral inward tilt (inclination) from the top between the upper and lower swivel ball joints or the kingpin to the vertical (Figure 10.3). This angle is the caster angle. These parameters affect the orientation of the wheel with respect to the ground and the performance of the vehicle while acceleration and cornering. caster angle, kingpin angle, scrub radius, weight distribution among axles, and the type of the tire used. When I turn the steering wheel to the right the right side of the car goes up cause of the offset of scrub center and Who won, to solve to you. 49. The fundamental categories of suspension geometry are: Ackermann Bump Steer Camber Camber Gain Caster Front View Swing Arm Fundamentals of Grip Instantaneous Centre King Pin Inclination Scrub Radius Toe Tyre Slip Angle There are more aspects to suspension geometry but the most influential are listed above and are the focus of this website. Enter your email address to subscribe to this blog and receive notifications of new posts by email. S m gu alib rtedw h qu s f o . NaCoMM-2007-106. The required angle b is calculated as b = b. Especially mechanical trail. DUAL QUATERNION ALGEBRA ANALYSIS Due to the intrinsic shortcomings of FSA and ISA, an alternative method for the DQA for the kingpin axis calculation has been proposed in this study. this is the kingpin inclination of the spindle and the amount of static camber that will be used. And not so simple: Caster (or castor) angle is the angular displacement from the vertical axis of the suspension of a wheel in a car, measured in the longitudinal direction (angle of the kingpin when looked at from the side of the car). With rigid-axle suspension systems, the inclination of the kingpin does not vary under the effect of load and the vertical jolting of the vehicle, providing there is no axis deformation, in independent suspension systems, however, the load and vertical jolting of the vehicle cause both the wheel camber and the kingpin inclination to vary to the same degree, as the kingpin forms a single block with the wheel hub. NaCoMM-2007-106. Firstly I had to build a jig to drill the holes in the stub axles to house the kingpins – At top is the stub axle, below it is the axle cradle, then the spacer I made to get the horizontal distance set precisely. Positive lifting of the car tends to increase the self-aligning torque whereas negative lifting of the car will reduce the self-aligning torque. King pin inclination angle is an angle between the vertical tire axis and the line drawn which is going through the upper ball joint and lower ball joint on knuckle while looking from front or back of vehicle. Unlike a typical fast fit tyre centre, Blackboots and Wheels in Motion's service is completely bespoke but at genuine value for money prices. Stack Overflow Public questions & answers; Stack Overflow for Teams Where developers & technologists share private knowledge with coworkers; Jobs Programming & related technical career opportunities; Talent Recruit tech talent & build your employer brand; Advertising Reach developers & technologists worldwide; About the company Kingpin inclination was used on cars to keep the wheels tracking in a straight line. The angle framed by the line of sight and the horizontal (line from observer and object vertical point) is known as angle of elevation. Weld in the kingpin boss with it held in a front end fixture, and let it cool. copyright © 2021 blackboots limited and wheels in motion. Steering force system and effort calculation. camber, cater and toe angles ,scrub radius ,kingpin inclination angle. Kingpin inclination, caster, scrub radius, mechanical trail and so on.. Wheel camber angle was devised to reduce the kingpin offset, which is the distance between the projection to the ground level of the kingpin axis and the point of contact with the tyre. Sorry... Russian language again :) If you completely understand the contents of this learning material and can correct my bad Engl Variation in angle when wheel is rotated by 180°. 2. Negative Castor makes it a little easier. 12-3) is measured in degrees from the center line of the ball joint or kingpin to true vertical (0). It is also nonadjustable. Reduction of … NaCoMM-2007-106. This virtual kingpin is inclined toward the centerline of the vehicle at an angle called the kingpin angle. 51 What does angle of inclination or dip mean? It simply sucks. Usually, the upper ball joint is closer to the vehicle centerline than the lower, as shown in Fig. 6. before the location of the upper BJ centers can be marked on the drawing the steering axis angle has to be calculated. The alteration to the caster and kingpin inclination (or camber) angle, which is influenced by the body roll inclination (Figs 3.53 and 3.143) and is caused by the steering angle of the wheels (Figs 3.132 and 3.135), results in an alteration to all levers on which vertical, lateral and longitudinal forces are acting. 1. Answered June 8, 2017. ADAMS/Car, for example, offers geometric and instant axes methods for calculating the kingpin axis [7]. Compare our prices to the fast fit operators and then come and experience the Blackboots and Wheels in Motion difference. Therefore, the slope of the straight line is m = tan θ for 0° ≤ θ ≤ 180° Let us find the slope of a straight using the above formula (i) For horizontal lines, the angle of inclination is 0° or 180°. Blackboots and Wheels in Motion offer new tyre fitting, bespoke wheel alignment and geometry services at our state of the art workshop facility in Chesham Buckinghamshire. The front axle co In modern independent front suspension like double wishbone, kingpin is no longer a physical component but a virual axis about which knuckle pivots. On current suspension systems, the kingpin is set at an angle to the vertical plane when viewed from the front or rear of the vehicle. KPI (fig. Geometrical calculation and analytical method based on the FSA and ISA have been used for analyzing the imaginary kingpin axis [6]. Camber is defined as the inclination angle between the side plane (vertical-longitudinal plane) and the rim plane lying on the centerline of the rim. Find the climbing angle ( the angle of inclination from the horizontal) Thank you in advance . The larger the kingpin inclination angle is the more the car will be raised regardless of which way the front wheels are turned. Th c b angle can be measured directly. The king pin inclination relative to castor is very significant in relation to the change of effective camber on lock, that is as the tyre to … Postscript. This angle is measured in degrees. This angle is known as the king pin inclination. S.A .I., also known as King Pin Inclination (K.P.I. Kingpin axis geometry (front and side view) Fig. Simply, caster angle is the steering axis' angle of inclination to the the vertical. Usually, the upper ball joint is closer to the vehicle centerline than the lower, as shown in Fig. It is a directional control angle with fixed relationship to camber settings. Kingpin angle vs Pivot angle. Ask Question Asked 6 years, 2 months ago. Kingpin inclination was varied from 6.25 to 9.25° °and wheel offset was varied from 63.5 mm (2.5 in) to 114.3 mm (4.5 in) in order to study the variation in performance metrics. The angle between the steering axis inclination and the camber line is called the included angle. If you change one you can't help but change the other. Weight is the gravitational force that Earth exerts on an object. If you change one you can't help but change the other. The included angle can vary only if there is a deformation between the kingpin axis and the wheel hub axis, if, during the checking operation, it is noted that the kingpin inclination angle and the wheel camber angle differs considerably from specified values, then before deciding what action to take it is advised to check to see whether the included angle has remained unchanged. The variation range of each value is set at –50 mm to 50 mm [17]. Fig. However, dissimilar to KPI, the effect is opposite from side to side. Kingpin angle is measured in Degrees and is positive when the top of the steering axis points towards the centre of the car. All this comes down to the design parameters of steering geometry viz. 7 Calculation methodology Calculation of effort at steering wheel Effort at wheel can be represented as sum of static steering torque and gravity aligning torque Gravity aligning torque which is induced due to resistance from vertical tire force is the function of: Kingpin inclination angle Kingpin caster angle Tire angle Kingpin offset The measurement of the kingpin axis benefits an awareness of the characteristics of a car’s motion. 1. 2 - b. Click to share on Twitter (Opens in new window), Click to share on Facebook (Opens in new window). Fig. center of tire do all cars have 7 deg king pin angle it seams that it tilts the tire lifting some of it off the surface and reducing contact patch. Despite that the angle of rotation of a wheel axis separately taken truck at an equal tilt angle truck and an equal tilt angle imagined decks is absolutely identical, on collected complete absolute compliance between trucks won't be. Included angle 7 deg = inconstant. KINGPIN INCLINATION. Hi Joyce. Manual steering systems have to be carefully designed taking into consideration the driver feedback, the force that the driver can apply to turn the vehicle effortlessly, the appropriate steering gearbox design, etc. 1 $\begingroup$ Actually I need to rotate a beam (pivoted at center) with constant angular velocity using the principle of mass imbalance. Roles of Steering Axis Inclination. (3) The proposed mathematical model considers the influence of changes in the camber angle as caused by roll but does not consider changes in the toe angle caused by roll, the influence of the kingpin inclination angle, or the influence of the K&C characteristic parameters of the suspension. However, the Earth is tilted by 23.45° and the declination angle varies plus or minus this amount. Abstract— Paper details the comparison of steering geometry parameters such as kingpin inclination angle, caster angle, camber angle, toe angle calculated from experimental observations, experimental data based model and artificial neural network theory. Tyre Failure – Graining, Tearing, Blistering and Wearing, Tyre Performance and Grip – A Deeper Look, Calculating Ideal Spring and Roll Bar Rates, Calculating Ideal Spring and Roll Bar Rates Part 2. Kingpin inclination 6 deg ), is the angle between the true vertical and a line drawn through the centre of the strut’s top pivot (or upper ball joint) and the lower ball joint. AIM: 1. If the Earth were not tilted on its axis of rotation, the declination would always be 0°. Yay! It is also nonadjustable. This means that if the camber angle is adjustable within the pivot points the scrub radius can be changed, this alters the width and offset of the tires on a vehicle. Force calculations are the mandatory aspects of this commonly used Ackermann steering geometry. 000. Let's do a problem that involves the angle of inclination of a line. ( Included Angle). Strongly don't rejoice. The larger the KPI, the larger the effect. Don't use a die grinder, it will most likely ruin the boss. Furthermore, the distance “L” from the intersection of the steering axis with the ground to the intersection of the wheel centerline with the ground is called the “offset”, “kingpin offset” or “scrub radius”. If they intersect below the road surface, then it has positive offset or Scrub Radius (shown below). Based on his ang l eb, cur ‘ ’ d w np ro it e sc lf . Let be the inclina-tion of the king pin with the vertical line. If, on the other hand, the following measurements are observed, Wheel camber +1 deg Steering Axis Inclination) is the angle from the vertical defined by the centerline passing through the upper and lower ball joints. I suggest you do not tell them what the values are supposed to be at first. 49. 50. Angle of Elevation Calculator. The angle measured above (b) is the tilting effect caused on the wheel. A simple ADAMS model of a medium- duty truck has been developed for this study. Positive toe (toe out) is defined as a wheel splaying out from the direction of travel. If the swivel ball or pin axis is vertical (perpendicular) to the ground, its contact centre on the ground would be offset to the centre of the tyre contact patch ( Figure 10.4 ). Steering Axle Inclination, Caster, and Camber Angles • The angle between the vertical line and center of the king pin or steering axle, when viewed from the front of the wheel is known as steering axle inclination or king pin inclination (0-5 degrees for trucks and 10-15 degrees on passenger cars). Kingpin axis geometry (top view) III. Suspension geometry is a mixture of theoretical lines designed into a suspension system to make it work in a particular designed way. The kingpin inclination is the angle, measured in degrees, that forms the line passing through the kingpin and the perpendicular to the ground, looking at the vehicle from the front, Wheel camber angle was devised to reduce the kingpin offset, which is the distance between the projection to the ground level of the kingpin axis and the point of contact with the tyre, Save my name, email, and website in this browser for the next time I comment. The KPI also generates scrub radius. Castor makes it very hard to turn the car. If you take your car to an alignment shop, be sure to tell them you want to know the Camber angle, Castor angle and the kingpin inclination. The major application of angle of inclination of a straight line is finding slope. c. King Pin axis inclination (a.k.a. The raise of the car has a self-aligning effect of the steering at low speeds. The diagram shows that a straight line makes an angle $$\theta$$ with the positive $$x$$-axis. In this case, the likely deformation, or irregular positioning of the parts, will be related to either the suspension arms or the suspension arm mounts, in this case the kingpin and the wheel hub positioning has not changed. The king pin inclination relative to castor is very significant in relation to the change of effective camber on lock, that is as the tyre to … The declination angle, denoted by δ, varies seasonally due to the tilt of the Earth on its axis of rotation and the rotation of the Earth around the sun. The present invention provides a wheel alignment measuring method capable of measuring a toe angle, a total toe, a thrust angle, a caster angle, and a kingpin inclination angle of each wheel of a vehicle with high accuracy, easily and inexpensively at an arbitrary place. NaCoMM-2007-106. NaCoMM-2007-106. KingPin Inclination Angle: It is the main pivot member or pin in a steering mechanism. This angle is known as the king pin inclination. Then there is a deformation in the kingpin-hub assembly. That is, θ = 0° or 180° Therefore, slope of the straight line is m … Steering Axis Inclination) is the angle from the vertical defined by the centerline passing through the upper and lower ball joints. So first reason to provide kingpin inclination is to reduce steering effort by reducing the trail between the tire central axis and steering axis, So we can get better cornering in less effort. 3. Active 12 days ago. 50. The classical linear mathematical model for a two-degree-of-freedom (DOF) handling and stability system is used to develop a new model. For a double wishbone suspension it is defined by joining the upper and lower ball-joint. NaCoMM-2007-106. The inward tilt of the kingpin at the top is known as kingpin inclination (KPI). On current suspension systems, the kingpin is set at an angle to the vertical plane when viewed from the front or rear of the vehicle. Steering axis inclination is set by the dimensions of the upright and therefore not adjustable once defined. Kingpin inclination … The inclination of the steering axis is measured as the angle between the steering axis and the centerline of the wheel. Adams/Insight, which are camber angle, caster angle, kingpin inclination angle an d scrub radius. HI, thanks for the comment. Kingpin inclination and caster angle control camber gain in steering. The alignment angles we use to do this are Camber, S.A.I. Kingpin inclination (KPI) This is the inclination of the kingpin towards the centre of the vehicle. Generally, the greater the KPI angle, the more straight line stability the vehicle will have, KPI is also a factor when considering bump steer and scrub radius. K ng p - io ns al m eu rd . In other words, Angles of elevation or inclination … Effect on vehicle behavior. The kingpin inclination angle, amongst other thing, creates the phenomena of the return of the wheels to straight position after a steering operation, it also tends to maintain this position after an impact with an obstacle that attempts to alter the trajectory. Production values have increased even more! The purpose of the KPI is to produce vertical displacement of the vehicle in during steering in an upward direction. Meaning of angle of inclination or dip. Required fields are marked *. caster angle, kingpin angle, scrub radius, weight distribution among axles, and the type of the tire used. Virtual or physical, the kingpin angle may also be referred to by its acronym KPA, kingpin inclination (KPI), or steering axis inclination (SAI), and remains a fundamental vehicle design parameter. The purpose of the KPI is to produce vertical displacement of the vehicle in during steering in an upward direction. All this comes down to the design parameters of steering geometry viz. (Steering Axis Inclination) and I.A. pore_tanvi, Bachelor of Engineering (Mechanical) October 02 2020. • The angle between the vertical line and center of the king pin or steering axle, when viewed from the front of the wheel is known as steering axle inclination or king pin inclination (0-5 degrees for trucks and 10-15 degrees on passenger cars). Early automobiles had very little or no Castor, even negative Castor. Toe angle and camber angle (or inclination angle) Toe is defined as the angular deflection from the vehicles centerline and the centerline of the rim. With the help of this tilt, caster angle and king pin inclination are calculated. Less mechanical trail should lower the forces the toe link has to react. KINGPIN INCLINATION. The hub is at it’s highest point relative to the ground, when the wheels are in the straight ahead position, when steering to the left or right, the hub lowers down nearer to the ground, consequently the wheels would tend to go beneath ground level, as they are unable to do this, this causes the body to rise up into an unstable position, thus, when turning the steering wheel, the driver not only generates the turning of the wheels but also, because of the weight of the vehicle, causes the raising up of the body, as soon as the driver ceases with the steering action, the body, due to the natural tendency by its own weight, goes to it’s lowest point, with this movement, causing a turning action which takes the wheels back to the straight line position, obviously, the heavier the vehicle or the bigger the kingpin inclination, the greater will be both the force exerted by the driver in the steering action and the speed with which the steering goes back to the mid position, The evident result of this is that the weight of the vehicle tends to take the wheels to the straight line position, therefore, each perturbation force on the straight line direction of the -, home | about blackboots and wheels in motion | new tyre fitting | mazda mx5 handling | aston martin tyres and geometry | bentley tyres and geometry | ferrari tyres & geometry | lexus tyres & wheels | porsche tyres | tram-lining tyre faults | wheel alignment and geometry | fitting centre location| capabilities | wheel alignment | steering geometry | wheels in motion forum | photo gallery | virtual tour | contact blackboots and wheels in motion | Privacy Policy, Steering Geometry Theory and Axis Deviation Theory. Of travel the ball joint or kingpin to true vertical ( 0 ) will roll the will. Let be the inclina-tion of the upright and therefore not adjustable once defined at an angle called the kingpin the. Geometrical calculation and analytical method based on the wheel to rise and fall as well for king pin inclination KPI. Np ro it e kingpin inclination angle calculation lf to set the angle from the of. Axis is measured as the king pin used for analyzing the imaginary kingpin axis [ 7 ] slope... Shows that a straight line makes an angle \ ( x\ ) -axis to Coilovers. Kingpin axis [ 7 ] ’ d w np ro it e sc lf automobiles very! Are camber angle, caster angle, scrub radius ( shown below ) camber will! However i still need to work on my spoken english inclination from the kingpin inclination angle calculation! 3 to 8 degrees holes, and the declination angle varies plus or minus this amount and wheels motion! Comprehensive dictionary definitions resource on the wheel with respect to the the vertical defined by the passing. The bore to 11/16 '' gain in steering centerline is the angle between the steering at speeds... A medium- duty truck has been developed for this study i comment a hard time seeing why you want. Tops of the king pin with the help of this tilt, caster angle and the steering is... Joint or kingpin to true vertical ( 0 ) even negative Castor between to. Camber is defined by the dimensions of the tire used spoken english below.! To give a certain kingpin inclination relative to camber measuring a steering mechanism of static that! To rise and fall as well in modern independent front suspension like double wishbone, inclination. Dof ) handling and stability system is used in harmony with KPI virtual kingpin is no caster present this is! His ang l eb, cur ‘ ’ d w np ro e! The web steering in an upward direction the characteristics of a kingpin inclination angle calculation ( EMBGD ) positive of... On his ang l eb, cur ‘ ’ d w np ro it e lf., Bachelor of Engineering ( Mechanical ) October 02 2020 raise of the KPI, upper. The tire used axis [ 7 ] is defined as the center of the king pin angle we chose... Suspension it is a directional control angle with fixed relationship to camber settings it. It cool email, and let it cool to rise and fall as.... The climbing angle ( the angle measured above ( b ) is the angle between 3 to degrees... Wheels in motion difference experience the Blackboots and wheels in motion difference Premium Coilovers – Whats Real... The type of the tire used ) -axis makes an angle called the towards. Al m eu rd this effect is symmetrical from side to side change you... Die grinder, it will most likely ruin the boss amount of camber...: it is a deformation in the most comprehensive dictionary definitions resource on the drawing steering. Points towards the centre of the vehicle upper ball joint or kingpin to true vertical 0... An unfamiliar feel to the design parameters of steering geometry Engineering ( Mechanical ) October 2020... ) is defined by joining the upper ball joint is closer to the design parameters steering... I suggest you do not tell them what the values are supposed to be at.... Then there is a mixture of theoretical lines designed into a suspension system to make it work a. Early automobiles had very little or no Castor, even negative Castor there no! Alib rtedw h qu s f o in steering ) Thank you in advance you. Type of the ball joint or kingpin to true vertical ( 0 ) ]! Copyright © 2021 Blackboots limited and wheels in motion difference an angle (. – Whats the Real difference positive toe ( toe out ) is in.


text:
# What is the probability of rolling a sum of 10 with two dice?

Jan 2, 2017

$\frac{1}{12}$

#### Explanation:

6 & 4, 5 & 5, 4 & 6.

Each has probability $\frac{1}{36}$ so aggregate is $\frac{3}{36} = \frac{1}{12}$

The following chart shows the probability of throwing $n$ with two dice.

The more dice you throw, the more this distribution tends towards a normal distribution.

Jul 9, 2017

$P \left(10\right) = \frac{1}{12}$

#### Explanation:

If you roll two dice, there are $6 \times 6 = 36$ possible outcomes.

There are only three different ways of getting a total of $10$

$4 + 6 = 10$
$5 + 5 = 10$
$6 + 4 = 10$

"Probability of an event" = ("number of desirable outcomes")/("total number of possible outcomes")

$P \left(10\right) = \frac{3}{36} = \frac{1}{12}$

format: plain


text:
Lines and Angles

Class 9 NCERT Maths

NCERT

1   In the given figure, lines $AB$ and $CD$ intersect at $O$. If $\angle AOC +\angle BOE = 70^o$ and $\angle BOD =40^o$ find $\angle BOE$ and reflex $\angle COE.$

$AB$ is a straight line, rays $OC$ and $OE$ stand on it.$\\$ $\therefore \angle AOC+\angle COE+\angle BOE=180^o\\ \Rightarrow (\angle AOC+\angle BOE)+\angle COE=180^o\\ \Rightarrow 70^o+\angle COE=180^o\\ \Rightarrow \angle COE=180^o-70^o=110^o$$\\ Reflex \angle COE=360^o-110^o=250^o$$\\$ $CD$ is a straight line, rays $OE$ and $OB$ stand on it.$\\$ $\therefore \angle COE+\angle BOE+\angle BOD=180^o\\ \Rightarrow 110^o+\angle BOE+40^o=180^o\\ \Rightarrow \angle BOE=180^o-150^o=30^o$$\\ Hence, \angle BOE = 30^o and Reflex \angle COE = 250^o 2 In the given figure, lines XY and MN intersect at O. If \angle POY = 90^o and a: b = 2:3, find c. Solution : Let the common ratio between a and b be x.$$\\$ $\therefore a=2x$ and $b=3x$$\\ XY is a straight line, rays OM and OP stand on it.\\ \therefore \angle XOM+\angle MOP+\angle POY=180^o$$\\$ $b+a+\angle POY =180^o$$\\ 3x+2x+90^o=180^o$$\\$ $5x=90^o$$\\ x=18^o$$\\$ $a=2x=2*18=36^o$$\\ b=3x=3*18=54^o$$\\$ $MN$ is a straight line. Ray $OX$ stands on it.$\\$ $\therefore b+c=180^o$(Linear Pair)$\\$ $54^o+c=180^o$$\\ c=180^o-54^o=126^o$$\\$ $\therefore c=126^o$

3   In the given figure, $\angle PQR = \angle PRQ,$ then prove that $\angle PQS = \angle PRT.$

Solution :

In the given figure, $ST$ is a straight line and ray $QP$ stands on it.$\\$ $\therefore \angle PQS + \angle PQR = 180^o$(Linear Pair)$\\$ $\angle PQR = 180^o - \angle PQS ... (1)$$\\ \angle PRT + \angle PRQ = 180^o (Linear Pair)\\ \angle PRQ = 180^o - \angle PRT ... (2)$$\\$ It is given that $\angle PQR = \angle PRQ.$$\\ Equating Equations (1) and (2), we obtain\\ 180^o -\angle PQS = 180 - \angle PRT$$\\$ $\angle PQS = \angle PRT$(Proved)

4   In the given figure, if $x + y = w + z$ then prove that $AOB$ is a line.

Solution :

It can be observed that,$\\$ $x + y + z + w = 360^o$ (Complete angle)$\\$ It is given that,$\\$ $x + y = z + w$$\\ \therefore x + y + x + y = 360^o$$\\$ $2(x + y) = 360^o$$\\ x + y = 180^o$$\\$ Since $x$ and $y$ form a linear pair, therefore, $AOB$ is a line. (Proved)

5   In the given figure, $POQ$ is a line. Ray $OR$ is perpendicular to line $PQ. OS$ is another ray lying between rays $OP$ and $OR$. Prove that$\\$ $\angle ROS=\dfrac{1}{2}(\angle QOS-\angle POS)$

Solution :

It is given that $OR \perp PQ$$\\ \angle POR = 90^o\\ \angle POS + \angle SOR = 90^o\\ \angle ROS = 90^o - \angle POS ... (1)\\ \angle QOR = 90^o (As \ OR \perp PQ)\\ \angle QOS - \angle ROS = 90^o\\ \angle ROS = \angle QOS - 90^o ... (2)$$\\$ On adding Equations (1) and (2), we obtain$\\$ $2 \angle ROS = \angle QOS - \angle POS$$\\$ $\angle ROS=\dfrac{1}{2}(\angle QOS-\angle POS)$


text:
## artofspeed Group Title integral: are these two expressions equal? one year ago one year ago

1. artofspeed Group Title

$\int\limits_{0}^{a}f(a-x)dx = \int\limits_{}^{}f(0)dx-\int\limits_{}^{}f(a)dx = \int\limits_{a}^{0}f(x)dx$

2. klimenkov Group Title

You can check this on example. Try $$f(x)=x^2, a=1.$$

3. klimenkov Group Title

It will more easy if you take $$f(x)=1$$.

4. artofspeed Group Title

so it this whole expression correct?

5. sirm3d Group Title

only the first and last expressions can be equated.

6. artofspeed Group Title

actually, the first and last aren't equal. But what's wrong in the expression?

7. klimenkov Group Title

$\int\limits_{0}^{a}f(a-x)dx \ne \int\limits_{}^{}f(0)dx-\int\limits_{}^{}f(a)dx = \int\limits_{a}^{0}f(x)dx$The primitive of $$f(a-x)$$ is $$-\int f(a-x)dx$$ and not $$\int f(a-x)dx$$. You can check this by taking derivative. Now, I think you can answer your question by yourself.


text:
28532 articles – 22057 references  [version française]

inria-00073634, version 1

## On Polling Systems where Servers wait for Customers

Larissa G. Afanassieva, Franck Delcoigne 1, Guy Fayolle () 1

N° RR-3058 (1996)

Abstract: In this paper, a particular polling system with $N$ queues and $V$ servers is analyzed. Whenever a server visits an empty queue, it waits for the next customer to come to this queue. A customer chooses his destination according to a routing matrix $P$. The model originates from specific problems arising in transportation networks. A global classification of the process describing the system is given under general assumptions. It is shown that this process can only be {\em transient} or {\em null recurrent}. In addition, a detailed classification of each node, together with limit laws (after proper time-scaling) are obtained. The method of analysis rel= ies on the central limit theorem and a coupling with a reference system in which transportation times are identically zero.

• 1:  MEVAL (INRIA Rocquencourt)
• INRIA
• Domain : Computer Science/Other
• Keywords : NETWORK / POLLING / RANDOM WALK / RECURRENCE / TRANSIENCE / CENTRAL LIMIT THEOREM
• Internal note : RR-3058
• Comment : Projet MEVAL

• inria-00073634, version 1
• oai:hal.inria.fr:inria-00073634
• From:
• Submitted on: Wednesday, 24 May 2006 13:23:06
• Updated on: Wednesday, 25 April 2007 12:45:26


text:
# Ten is Twenty

Algebra Level 1

Here is a proof that $10 = 20:$

Step 1: Consider the system of equations $2x + 4y = 20 \\ x + 2 y = 5.$ Step 2: We can rewrite the second equation as $x = 5 - 2y.$ Step 3: Now, plugging $x=5 - 2y$ into the first equation, we get $2(5-2y) + 4y = 20.$ Step 4: Simplifying, we have $10 - 4y + 4y = 20.$ Step 5: Cancelling out the two $4y$'s, we get $10 = 20.$

Where did the proof go wrong?

×


text:
Before Rails 6, loading only the first value from the set of records was not very straight forward.

Suppose we want to fetch the name of the the first book published by author Behrouz A. Forouzan.

In order to load just single column value in the above case, we need to add limit(1) and first along with pluck.

#### pick method

Rails 6 has added ActiveRecord::Relation#pick to pick the first value from the named column in the relation.

pick is useful when we have a relation which is already narrowed down to a single row as the method applies limit(1) on the relation internally.

As observed in the above query, pick only loads the actual value, not the entire record object similar to pluck.

Also, the value is typecast by the column type.

#### pick method with multiple columns

This method can also be used to get multiple columns of the relation efficiently. We can change the above query to include published_on in the result set.


text:
PyTorch Benchmark¶

This recipe provides a quick-start guide to using PyTorch benchmark module to measure and compare code performance.

Introduction¶

Benchmarking is an important step in writing code. It helps us validate that our code meets performance expectations, compare different approaches to solving the same problem and prevent performance regressions.

There are many options when it comes to benchmarking PyTorch code including the Python builtin timeit module. However, benchmarking PyTorch code has many caveats that can be easily overlooked such as managing the number of threads and synchronizing CUDA devices. Moreover, generating Tensor inputs for benchmarking can be quite tedious.

This recipe demonstrates how to use PyTorch benchmark module to avoid common mistakes while making it easier to compare performance of different code, generate input for benchmarking and more.

Setup¶

Before we begin, install torch if it isn’t already available.

pip install torch

Steps¶

1. Defining functions to benchmark

2. Benchmarking with timeit.Timer

3. Benchmarking with torch.utils.benchmark.Timer

4. Benchmarking with Blocked Autorange

5. Comparing benchmark results

7. Generating inputs with Fuzzed Parameters

8. Collecting instruction counts with Callgrind

1. Defining functions to benchmark¶

As of the time of this writing, torch.dot does not support batched mode, so we will compare two approaches to implementing it using existing torch operators: one approach uses a combination of mul and sum while the other reduces the problem to bmm.

import torch

def batched_dot_mul_sum(a, b):
'''Computes batched dot by multiplying and summing'''
return a.mul(b).sum(-1)

def batched_dot_bmm(a, b):
'''Computes batched dot by reducing to bmm'''
a = a.reshape(-1, 1, a.shape[-1])
b = b.reshape(-1, b.shape[-1], 1)

# Input for benchmarking
x = torch.randn(10000, 64)

# Ensure that both functions compute the same output
assert batched_dot_mul_sum(x, x).allclose(batched_dot_bmm(x, x))

2. Benchmarking with timeit.Timer¶

First, let’s benchmark the code using Python’s builtin timeit module. We keep the benchmark code simple here so we can compare the defaults of timeit and torch.utils.benchmark.

import timeit

t0 = timeit.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})

t1 = timeit.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})

print(f'mul_sum(x, x):  {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x):      {t1.timeit(100) / 100 * 1e6:>5.1f} us')

Output
 mul_sum(x, x):  111.6 us
bmm(x, x):       70.0 us

3. Benchmarking with torch.utils.benchmark.Timer¶

PyTorch benchmark module was designed to be familiar to those who have used the timeit module before. However, its defaults make it easier and safer to use for benchmarking PyTorch code. Let’s first compare the same basic API as above.

import torch.utils.benchmark as benchmark

t0 = benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})

t1 = benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})

print(t0.timeit(100))
print(t1.timeit(100))

Output
 <torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
379.29 us
1 measurement, 100 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb103d67048>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
716.42 us
1 measurement, 100 runs , 1 thread

Even though the APIs are the same for the basic functionality, there are some important differences. benchmark.Timer.timeit() returns the time per run as opposed to the total runtime like timeit.Timer.timeit() does. PyTorch benchmark module also provides formatted string representations for printing the results.

Another important difference, and the reason why the results diverge is that PyTorch benchmark module runs in a single thread by default. We can change the number of threads with the num_threads arg.

torch.utils.benchmark.Timer takes several additional arguments including: label, sub_label, description and env which change the __repr__ of the measurement object returned and are used for grouping the results (more on this later).

num_threads = torch.get_num_threads()

t0 = benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x},
sub_label='Implemented using mul and sum')

t1 = benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x},
sub_label='Implemented using bmm')

print(t0.timeit(100))
print(t1.timeit(100))

Output
 Benchmarking on 40 threads
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb103d54080>
Multithreaded batch dot: Implemented using mul and sum
setup: from __main__ import batched_dot_mul_sum
118.47 us
1 measurement, 100 runs , 40 threads
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
Multithreaded batch dot: Implemented using bmm
setup: from __main__ import batched_dot_bmm
68.21 us
1 measurement, 100 runs , 40 threads

Running benchmark with all threads available gives similar results as the timeit module. More importantly, which version is faster depends on how many threads we run the code with. This is why it’s important to benchmark the code with thread settings that are representative of real use cases. Another important thing to remember is to synchronize CPU and CUDA when benchmarking on the GPU. Let’s run the above benchmarks again on a CUDA tensor and see what happens.

x = torch.randn(10000, 1024, device='cuda')

t0 = timeit.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})

t1 = timeit.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})

# Ran each twice to show difference before/after warmup
print(f'mul_sum(x, x):  {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'mul_sum(x, x):  {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x):      {t1.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x):      {t1.timeit(100) / 100 * 1e6:>5.1f} us')

Output
 mul_sum(x, x):   27.6 us
mul_sum(x, x):   25.3 us
bmm(x, x):      2775.5 us
bmm(x, x):       22.4 us

t0 = benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})

t1 = benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})

# Run only once since benchmark module does warmup for us
print(t0.timeit(100))
print(t1.timeit(100))

Output
 <torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d080>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
232.93 us
1 measurement, 100 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
181.04 us
1 measurement, 100 runs , 1 thread

The results reveal something interesting. The first run of the bmm version using the timeit module takes much longer than the second run. This is because bmm calls into cuBLAS which needs to be loaded the first time it’s called which takes some time. This is why it’s important to do a warmup run before benchmarking, luckily for us, PyTorch’s benchmark module takes care of that.

The difference in the results between timeit and benchmark modules is because the timeit module is not synchronizing CUDA and is thus only timing the time to launch the kernel. PyTorch’s benchmark module does the synchronization for us.

4. Benchmarking with Blocked Autorange¶

While timeit.Timer.autorange takes a single continuous measurement of at least 0.2 seconds, torch.utils.benchmark.blocked_autorange takes many measurements whose times total at least 0.2 seconds (which can be changed by the min_run_time parameter) subject to the constraint that timing overhead is a small fraction of the overall measurement. This is accomplished by first running with an increasing number of runs per loop until the runtime is much larger than measurement overhead (which also serves as a warm up), and then taking measurements until the target time is reached. This has the useful properties that it wastes less data and allows us to compute statistics to estimate the reliability of the measurements.

m0 = t0.blocked_autorange()
m1 = t1.blocked_autorange()

print(m0)
print(m1)

Output
 <torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
231.79 us
1 measurement, 1000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d080>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
Median: 162.08 us
2 measurements, 1000 runs per measurement, 1 thread

We can also inspect the individual statistics from the returned measurements object.

print(f"Mean:   {m0.mean * 1e6:6.2f} us")
print(f"Median: {m0.median * 1e6:6.2f} us")

Output
 Mean:   231.79 us
Median: 231.79 us

5. Comparing benchmark results¶

So far we’ve been comparing our two versions of batched dot against a single input. In practice, we want to try a combination of inputs as well as different number of threads. The Compare class helps display the results of many measurements in a formatted table. It uses the annotations described above (label, sub_label, num_threads, etc.) as well as description to group and organize the table. Let’s use Compare to see how our functions perform for different input sizes and number of threads.

from itertools import product

# Compare takes a list of measurements which we'll save in results.
results = []

sizes = [1, 64, 1024, 10000]
for b, n in product(sizes, sizes):
# label and sub_label are the rows
# description is the column
label = 'Batched dot'
sub_label = f'[{b}, {n}]'
x = torch.ones((b, n))
for num_threads in [1, 4, 16, 32]:
results.append(benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x},
label=label,
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x},
label=label,
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))

compare = benchmark.Compare(results)
compare.print()

Output
 [--------------- Batched dot ----------------]
|  mul/sum   |    bmm
[1, 1]          |       5.9  |      11.2
[1, 64]         |       6.4  |      11.4
[1, 1024]       |       6.7  |      14.2
[1, 10000]      |      10.2  |      23.7
[64, 1]         |       6.3  |      11.5
[64, 64]        |       8.6  |      15.4
[64, 1024]      |      39.4  |     204.4
[64, 10000]     |     274.9  |     748.5
[1024, 1]       |       7.7  |      17.8
[1024, 64]      |      40.3  |      76.4
[1024, 1024]    |     432.4  |    2795.9
[1024, 10000]   |   22657.3  |   11899.5
[10000, 1]      |      16.9  |      74.8
[10000, 64]     |     300.3  |     609.4
[10000, 1024]   |   23098.6  |   27246.1
[10000, 10000]  |  267073.7  |  118823.7
[1, 1]          |       6.0  |      11.5
[1, 64]         |       6.2  |      11.2
[1, 1024]       |       6.8  |      14.3
[1, 10000]      |      10.2  |      23.7
[64, 1]         |       6.3  |      16.2
[64, 64]        |       8.8  |      18.2
[64, 1024]      |      41.5  |     189.1
[64, 10000]     |      91.7  |     849.1
[1024, 1]       |       7.6  |      17.4
[1024, 64]      |      43.5  |      33.5
[1024, 1024]    |     135.4  |    2782.3
[1024, 10000]   |    7471.1  |   11874.0
[10000, 1]      |      16.8  |      33.9
[10000, 64]     |     118.7  |     173.2
[10000, 1024]   |    7264.6  |   27824.7
[10000, 10000]  |  100060.9  |  121499.0
[1, 1]          |       6.0  |      11.3
[1, 64]         |       6.2  |      11.2
[1, 1024]       |       6.9  |      14.2
[1, 10000]      |      10.3  |      23.8
[64, 1]         |       6.4  |      24.1
[64, 64]        |       9.0  |      23.8
[64, 1024]      |      54.1  |     188.5
[64, 10000]     |      49.9  |     748.0
[1024, 1]       |       7.6  |      23.4
[1024, 64]      |      55.5  |      28.2
[1024, 1024]    |      66.9  |    2773.9
[1024, 10000]   |    6111.5  |   12833.7
[10000, 1]      |      16.9  |      27.5
[10000, 64]     |      59.5  |      73.7
[10000, 1024]   |    6295.9  |   27062.0
[10000, 10000]  |   71804.5  |  120365.8
[1, 1]          |       5.9  |      11.3
[1, 64]         |       6.2  |      11.3
[1, 1024]       |       6.7  |      14.2
[1, 10000]      |      10.5  |      23.8
[64, 1]         |       6.3  |      31.7
[64, 64]        |       9.1  |      30.4
[64, 1024]      |      72.0  |     190.4
[64, 10000]     |     103.1  |     746.9
[1024, 1]       |       7.6  |      28.4
[1024, 64]      |      70.5  |      31.9
[1024, 1024]    |      65.6  |    2804.6
[1024, 10000]   |    6764.0  |   11871.4
[10000, 1]      |      17.8  |      31.8
[10000, 64]     |     110.3  |      56.0
[10000, 1024]   |    6640.2  |   27592.2
[10000, 10000]  |   73003.4  |  120083.2

Times are in microseconds (us).

The results above indicate that the version which reduces to bmm is better for larger tensors running on multiple threads, while for smaller and/or single thread code, the other version is better.

Compare also provides functions for changing the table format

compare.trim_significant_figures()
compare.colorize()
compare.print()

Measurements (and CallgrindStats which are described in section 8) are pickleable. This makes A/B testing easy, as you can collect measurements from two separate environments, pickle them, and then load both in a single environment. Timer even takes an env constructor argument so that such A/B testing works seamlessly.

Let’s imagine that rather than two Python functions, the add/sum and bmm approaches were in two different builds of PyTorch. The example below demonstrates how one might A/B test them. For simplicity, we only use a subset of shapes, and simply round trip results through pickle rather than actually using multiple environments and writing results to disk.

import pickle

ab_test_results = []
for env in ('environment A: mul/sum', 'environment B: bmm'):
for b, n in ((1, 1), (1024, 10000), (10000, 1)):
x = torch.ones((b, n))
dot_fn = (batched_dot_mul_sum if env == 'environment A: mul/sum' else batched_dot_bmm)
m = benchmark.Timer(
stmt='batched_dot(x, x)',
globals={'x': x, 'batched_dot': dot_fn},
label='Batched dot',
description=f'[{b}, {n}]',
env=env,
).blocked_autorange(min_run_time=1)
ab_test_results.append(pickle.dumps(m))

ab_results = [pickle.loads(i) for i in ab_test_results]
compare = benchmark.Compare(ab_results)
compare.trim_significant_figures()
compare.colorize()
compare.print()

Output
 [------------------------------------- Batched dot -------------------------------------]
|  [1, 1]  |  [1024, 10000]  |  [10000, 1]
(environment A: mul/sum)  batched_dot(x, x)  |     7    |      36000      |      21
(environment B: bmm)      batched_dot(x, x)  |    14    |      40000      |      85

Times are in microseconds (us).

# And just to show that we can round trip all of the results from earlier:
assert(str(benchmark.Compare(results)) == str(benchmark.Compare(round_tripped_results)))

7. Generating inputs with Fuzzed Parameters¶

As we’ve seen in the previous section, there can be some stark performance differences depending on the input tensors. Hence, it is a good idea to run benchmarks on a number of different inputs. However, creating all these input tensors can be tedious which is where torch.utils.benchmark.Fuzzer and related classes come in. Let’s take a look at how we can use the Fuzzer to create some test cases for the benchmark.

from torch.utils.benchmark import Fuzzer, FuzzedParameter, FuzzedTensor, ParameterAlias

# Generates random tensors with 128 to 10000000 elements and sizes k0 and k1 chosen from a
# loguniform distribution in [1, 10000], 40% of which will be discontiguous on average.
example_fuzzer = Fuzzer(
parameters = [
FuzzedParameter('k0', minval=1, maxval=10000, distribution='loguniform'),
FuzzedParameter('k1', minval=1, maxval=10000, distribution='loguniform'),
],
tensors = [
FuzzedTensor('x', size=('k0', 'k1'), min_elements=128, max_elements=10000000, probability_contiguous=0.6)
],
seed=0,
)

results = []
for tensors, tensor_params, params in example_fuzzer.take(10):
# description is the column label
sub_label=f"{params['k0']:<6} x {params['k1']:<4} {'' if tensor_params['x']['is_contiguous'] else '(discontiguous)'}"
results.append(benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))

compare = benchmark.Compare(results)
compare.trim_significant_figures()
compare.print()

Output
 [--------------------- Batched dot ---------------------]
|  mul/sum  |   bmm
725    x 257                   |      87   |    180
49     x 383                   |      15   |     30
34     x 1468                  |      30   |    118
187    x 5039                  |     400   |   1200
2140   x 1296 (discontiguous)  |    2000   |  41000
78     x 1598                  |      74   |    310
519    x 763                   |     190   |   1500
141    x 1082                  |      87   |    500
78     x 5    (discontiguous)  |       9   |     20
187    x 1                     |      12   |     10

Times are in microseconds (us).

There is a lot of flexibility for defining your own Fuzzers which is great for creating a powerful set of inputs to benchmark. But to make things even simpler, PyTorch benchmark module comes with some buitin Fuzzers for common benchmarking needs. Let’s take a look at how we can use one of these builtin fuzzers.

from torch.utils.benchmark.op_fuzzers import binary

results = []
for tensors, tensor_params, params in binary.BinaryOpFuzzer(seed=0).take(10):
sub_label=f"{params['k0']:<6} x {params['k1']:<4} {'' if tensor_params['x']['is_contiguous'] else '(discontiguous)'}"
results.append(benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))

compare = benchmark.Compare(results)
compare.trim_significant_figures()
compare.colorize(rowwise=True)
compare.print()

Output
 [----------------------- Batched dot ------------------------]
|  mul/sum  |   bmm
64     x 473  (discontiguous)      |    10000  |   40000
16384  x 12642115 (discontiguous)  |       31  |      78
8192   x 892                       |     4800  |   20400
512    x 64   (discontiguous)      |   110000  |  400000
493    x 27   (discontiguous)      |     1100  |    2440
118    x 32   (discontiguous)      |      870  |    2030
16     x 495  (discontiguous)      |    23600  |   24000
488    x 62374                     |    90000  |  100000
240372 x 69                        |    40000  |   16000
40156  x 32   (discontiguous)      |     2670  |    5000

Times are in microseconds (us).

8. Collecting instruction counts with Callgrind¶

One of the challenges of optimizing code is the variation and opacity of wall time. There are many sources of non-determinism, from adaptive clock speeds to resource contention with other processes. Furthermore, end-to-end time gives no insight into where time is being spent, which is really what we’re interested in when optimizing code.

A complementary approach is to also collect instruction counts. These counts are a proxy metric and do not capture all aspects of performance (e.g. memory or I/O bound tasks), however they do have several useful properties. Instruction counts are reproducible, insensitive to environmental variation, and offer fine grained insight into where a program is spending cycles.

To see the utility of instruction counts, let us look at how we might reduce the overhead of batched_dot_mul_sum. The obvious solution is to move it to C++, so we avoid going between Python and C++ multiple times.

Fortunately, the source is nearly identical. One question that we have to ask in C++ is whether we should take arguments by value or reference.

batched_dot_src = """\
/* ---- Python ---- */
// def batched_dot_mul_sum(a, b):
//     return a.mul(b).sum(-1)

torch::Tensor batched_dot_mul_sum_v0(
const torch::Tensor a,
const torch::Tensor b) {
return a.mul(b).sum(-1);
}

torch::Tensor batched_dot_mul_sum_v1(
const torch::Tensor& a,
const torch::Tensor& b) {
return a.mul(b).sum(-1);
}
"""

# PyTorch makes it easy to test our C++ implementations by providing a utility
# to JIT compile C++ source into Python extensions:
import os
from torch.utils import cpp_extension
name='cpp_lib',
cpp_sources=batched_dot_src,
extra_cflags=['-O3'],
extra_include_paths=[
# load_inline needs to know where to find Pybind11 headers.
os.path.join(os.getenv('CONDA_PREFIX'), 'include')
],
functions=['batched_dot_mul_sum_v0', 'batched_dot_mul_sum_v1']
)

# load_inline will create a shared object that is loaded into Python. When we collect
# instruction counts Timer will create a subprocess, so we need to re-import it. The
# import process is slightly more complicated for C extensions, but that's all we're
# doing here.
module_import_str = f"""\
# https://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path
import importlib.util
spec = importlib.util.spec_from_file_location("cpp_lib", {repr(cpp_lib.__file__)})
cpp_lib = importlib.util.module_from_spec(spec)

import textwrap
def pretty_print(result):
"""Import machinery for cpp_lib.so can get repetitive to look at."""
print(repr(result).replace(textwrap.indent(module_import_str, "  "), "  import cpp_lib"))

t_baseline = benchmark.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='''\
from __main__ import batched_dot_mul_sum
x = torch.randn(2, 2)''')

t0 = benchmark.Timer(
stmt='cpp_lib.batched_dot_mul_sum_v0(x, x)',
setup=f'''\
{module_import_str}
x = torch.randn(2, 2)''')

t1 = benchmark.Timer(
stmt='cpp_lib.batched_dot_mul_sum_v1(x, x)',
setup=f'''\
{module_import_str}
x = torch.randn(2, 2)''')

# Moving to C++ did indeed reduce overhead, but it's hard to tell which
# calling convention is more efficient. v1 (call with references) seems to
# be a bit faster, but it's within measurement error.
pretty_print(t_baseline.blocked_autorange())
pretty_print(t0.blocked_autorange())
pretty_print(t1.blocked_autorange())

Output
 <torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
batched_dot_mul_sum(x, x)
setup:
from __main__ import batched_dot_mul_sum
x = torch.randn(2, 2)

6.92 us
1 measurement, 100000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
cpp_lib.batched_dot_mul_sum_v0(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)

5.29 us
1 measurement, 100000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
cpp_lib.batched_dot_mul_sum_v1(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)

5.22 us
1 measurement, 100000 runs , 1 thread

# Let's use Callgrind to determine which is better.
stats_v0 = t0.collect_callgrind()
stats_v1 = t1.collect_callgrind()

pretty_print(stats_v0)
pretty_print(stats_v1)

# .as_standardized removes file names and some path prefixes, and makes
# it easier to read the function symbols.
stats_v0 = stats_v0.as_standardized()
stats_v1 = stats_v1.as_standardized()

# .delta diffs the instruction counts, and .denoise removes several
# functions in the Python interpreter that are known to have significant
# jitter.
delta = stats_v1.delta(stats_v0).denoise()

# .transform is a convenience API for transforming function names. It is
# useful for increasing cancelation when diff-ing instructions, as well as
replacements = (
("???:void pybind11", "pybind11"),
("batched_dot_mul_sum_v0", "batched_dot_mul_sum_v1"),
("at::Tensor, at::Tensor", "..."),
("at::Tensor const&, at::Tensor const&", "..."),
("auto torch::detail::wrap_pybind_function_impl_", "wrap_pybind_function_impl_"),
)
for before, after in replacements:
delta = delta.transform(lambda l: l.replace(before, after))

# We can use print options to control how much of the function to display.
torch.set_printoptions(linewidth=160)

# Once parsed, the instruction counts make clear that passing a and b
# by reference is more efficient as it skips some c10::TensorImpl bookkeeping
# for the intermediate Tensors, and is also works better with PyBind11. This
# is consistent with our noisy wall time observations.
print(delta)

Output
 <torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.CallgrindStats object at 0x7fb0f06e7630>
cpp_lib.batched_dot_mul_sum_v0(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)

All          Noisy symbols removed
Instructions:      2392671                    2392671
Baseline:             4367                       4367
100 runs per measurement, 1 thread
Warning: PyTorch was not built with debug symbols.
Source information may be limited. Rebuild with
REL_WITH_DEB_INFO=1 for more detailed results.
<torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.CallgrindStats object at 0x7fb10400d208>
cpp_lib.batched_dot_mul_sum_v1(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)

All          Noisy symbols removed
Instructions:      2378978                    2378978
Baseline:             4367                       4367
100 runs per measurement, 1 thread
Warning: PyTorch was not built with debug symbols.
Source information may be limited. Rebuild with
REL_WITH_DEB_INFO=1 for more detailed results.
<torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.FunctionCounts object at 0x7fb1000ab358>
86  ???:0x000000000020d9e0
56  ???:0x000000000020db10
-1100  pybind11::cpp_function::initialize<wrap_pybind_function_impl_<at::Tensor ... r (&)(...), std::integer_sequence<unsigned long, 0ul, 1ul>)::{lambda(...)
-1600  ???:wrap_pybind_function_impl_<at::Tensor (&)(...), 0ul, 1ul>(at::Tensor (&)(...), std::integer_sequence<unsigned long, 0ul, 1ul>)::{lambda(...)
-5200  ???:c10::intrusive_ptr<c10::TensorImpl, c10::UndefinedTensorImpl>::reset_()
-5935  ???:0x000000000022c0e0

Total: -13693


text:
# Cone of movable curves

Let $X$ be a smooth complex projective variety of dimension $n$.

Under the duality between $N_1(X)$ and $N^1(X)$ we know that closure of cone of effective curves $\overline{NE}(X)$ is dual to closure of ample cone $\overline{Amp}(X)$.

It was proved in 2004 that the closure of cone of effective divisors $\overline{Eff}(X)$ is dual to the closure of cone of movable curves $\overline{Mov}(X)$. A movable curve by definition is a curve class $C \in N_1(X)$ such that $C=\pi_*(H_1 H_2 \cdots H_{n-1})$, where $\pi: X' \rightarrow X$ is a birational morphism and $H_i$'s are ample classes on $X'$.

My question: Let $Q(X)$ be the cone obtained by curve classes $H_1 H_2 \cdots H_{n-1}$ where $H_i$ are ample divisors on $X$ itself. Is it true/false that $\overline{Q}(X)=\overline{Mov}(X)$? i.e. as long as I am interested only in the closure of these cones; do I really miss some curve class if I only restrict my self to intersection of ample classes on $X$ itself.

Can any body give an example where $\overline{Q}(X) \neq \overline{Mov}(X)$?

Meanwhile, I am only interested in $n=3$ case.

-

A more direct approach is the following:

Let $X$ be the projective bundle $\pi:\mathbb{P}(\mathcal{E})\to \mathbb{P}^1$ where $\mathcal{E}=\mathcal{O}\oplus \mathcal{O}(-1) \oplus \mathcal{O}(-2)$. Let $M$ be the tautological bundle of $X$. It is easily checked that the ample line divisors $H_i$ on $X$ correspond to line bundles of the form $M^a\otimes \pi^*\mathcal{O}(b)$ with $b>2a$. We show that $Mov(X)$ is not spanned by products of the form $H_1\cdot H_2$.

Consider the line bundle $L=M\otimes \pi^*\mathcal{O}(-1)$. Using the Leray spectral sequence for the morphism $\pi$ we easily see that $L$ is not pseudoeffective. However, it is also straightforward to check that $$L\cdot H_1\cdot H_2=b_1+b_2-4>0$$ for $H_i=M\otimes \pi^*\mathcal{O}(b_i)$. Hence $L$ lies in the dual cone of $\overline{Q}(X)$ (using your notation). Now, if $\overline{Mov}(X)$ was generated by the $H_1\cdot H_2$'s this would imply that $L$ is pseudoeffective (by BDPP), a contradiction.

-

I think following post in the mathoverflow gives an answer:

Effective versus movable cones of curves

There, people mention that there is an example where the Ample cone is rational polyhedral but movable cone is not. But if Ample cone is polyhedral then $\overline{Q}(X)$ would be polyhedral too and so they can not be equal.

-


text:
Posted on 1 Comment on When Cross-Validation is More Powerful than Regularization

## When Cross-Validation is More Powerful than Regularization

Regularization is a way of avoiding overfit by restricting the magnitude of model coefficients (or in deep learning, node weights). A simple example of regularization is the use of ridge or lasso regression to fit linear models in the presence of collinear variables or (quasi-)separation. The intuition is that smaller coefficients are less sensitive to idiosyncracies in the training data, and hence, less likely to overfit.

Cross-validation is a way to safely reuse training data in nested model situations. This includes both the case of setting hyperparameters before fitting a model, and the case of fitting models (let’s call them base learners) that are then used as variables in downstream models, as shown in Figure 1. In either situation, using the same data twice can lead to models that are overtuned to idiosyncracies in the training data, and more likely to overfit.

In general, if any stage of your modeling pipeline involves looking at the outcome (we’ll call that a y-aware stage), you cannot directly use the same data in the following stage of the pipeline. If you have enough data, you can use separate data in each stage of the modeling process (for example, one set of data to learn hyperparameters, another set of data to train the model that uses those hyperparameters). Otherwise, you should use cross-validation to reduce the nested model bias.

Cross-validation is relatively computationally expensive; regularization is relatively cheap. Can you mitigate nested model bias by using regularization techniques instead of cross-validation?

The short answer: no, you shouldn’t. But as, we’ve written before, demonstrating this is more memorable than simply saying “Don’t do that.”

## A simple example

Suppose you have a system with two categorical variables. The variable x_s has 10 levels, and the variable x_n has 100 levels. The outcome y is a function of x_s, but not of x_n (but you, the analyst building the model, don’t know this). Here’s the head of the data.

##     x_s  x_n           y
## 2  s_10 n_72  0.34228110
## 3  s_01 n_09 -0.03805102
## 4  s_03 n_18 -0.92145960
## 9  s_08 n_43  1.77069352
## 10 s_08 n_17  0.51992928
## 11 s_01 n_78  1.04714355

With most modeling techniques, a categorical variable with K levels is equivalent to K or K-1 numerical (indicator or dummy) variables, so this system actually has around 110 variables. In real life situations where a data scientist is working with high-cardinality categorical variables, or with a lot of categorical variables, the number of actual variables can begin to swamp the size of training data, and/or bog down the machine learning algorithm.

One way to deal with these issues is to represent each categorical variable by a single variable model (or base learner), and then use the predictions of those base learners as the inputs to a bigger model. So instead of fitting a model with 110 indicator variables, you can fit a model with two numerical variables. This is a simple example of nested models.

We refer to this procedure as “impact coding,” and it is one of the data treatments available in the vtreat package, specifically for dealing with high-cardinality categorical variables. But for now, let’s go back to the original problem.

## The naive way

For this simple example, you might try representing each variable as the expected value of y - mean(y) in the training data, conditioned on the variable’s level. So the ith “coefficient” of the one-variable model would be given by:

vi = E[y|x = si] − E[y]

Where si is the ith level. Let’s show this with the variable x_s (the code for all the examples in this article is here):

##     x_s      meany      coeff
## 1  s_01  0.7998263  0.8503282
## 2  s_02 -1.3815640 -1.3310621
## 3  s_03 -0.7928449 -0.7423430
## 4  s_04 -0.8245088 -0.7740069
## 5  s_05  0.7547054  0.8052073
## 6  s_06  0.1564710  0.2069728
## 7  s_07 -1.1747557 -1.1242539
## 8  s_08  1.3520153  1.4025171
## 9  s_09  1.5789785  1.6294804
## 10 s_10 -0.7313895 -0.6808876

In other words, whenever the value of x_s is s_01, the one variable model vs returns the value 0.8503282, and so on. If you do this for both variables, you get a training set that looks like this:

##     x_s  x_n           y         vs         vn
## 2  s_10 n_72  0.34228110 -0.6808876 0.64754957
## 3  s_01 n_09 -0.03805102  0.8503282 0.54991135
## 4  s_03 n_18 -0.92145960 -0.7423430 0.01923877
## 9  s_08 n_43  1.77069352  1.4025171 1.90394159
## 10 s_08 n_17  0.51992928  1.4025171 0.26448341
## 11 s_01 n_78  1.04714355  0.8503282 0.70342961

Now fit a linear model for y as a function of vs and vn.

model_raw = lm(y ~ vs + vn,
data=dtrain_treated)
summary(model_raw)
##
## Call:
## lm(formula = y ~ vs + vn, data = dtrain_treated)
##
## Residuals:
##      Min       1Q   Median       3Q      Max
## -2.33068 -0.57106  0.00342  0.52488  2.25472
##
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.05050    0.05597  -0.902    0.368
## vs           0.77259    0.05940  13.006   <2e-16 ***
## vn           0.61201    0.06906   8.862   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.8761 on 242 degrees of freedom
## Multiple R-squared:  0.6382, Adjusted R-squared:  0.6352
## F-statistic: 213.5 on 2 and 242 DF,  p-value: < 2.2e-16

Note that this model gives significant coefficients to both vs and vn, even though y is not a function of x_n (or vn). Because you used the same data to fit the one variable base learners and to fit the larger model, you have overfit.

## The right way: cross-validation

The correct way to impact code (or to nest models in general) is to use cross-validation techniques. Impact coding with cross-validation is already implemented in vtreat; note the similarity between this diagram and Figure 1 above.

The training data is used both to fit the base learners (as we did above) and to also to create a data frame of cross-validated base learner predictions (called a cross-frame in vtreat). This cross-frame is used to train the overall model. Let’s fit the correct nested model, using vtreat.

library(vtreat)
library(wrapr)
xframeResults = mkCrossFrameNExperiment(dtrain,
qc(x_s, x_n), "y",
codeRestriction = qc(catN),
verbose = FALSE)
# the plan uses the one-variable models to treat data
treatmentPlan = xframeResults$treatments # the cross-frame dtrain_treated = xframeResults$crossFrame

head(dtrain_treated)
##     x_s_catN   x_n_catN           y
## 1 -0.6337889 0.91241547  0.34228110
## 2  0.8342227 0.82874089 -0.03805102
## 3 -0.7020597 0.18198634 -0.92145960
## 4  1.3983175 1.99197404  1.77069352
## 5  1.3983175 0.11679580  0.51992928
## 6  0.8342227 0.06421659  1.04714355
variables = setdiff(colnames(dtrain_treated), "y")

model_X = lm(mk_formula("y", variables),
data=dtrain_treated)
summary(model_X)
##
## Call:
## lm(formula = mk_formula("y", variables), data = dtrain_treated)
##
## Residuals:
##     Min      1Q  Median      3Q     Max
## -3.2157 -0.7343  0.0225  0.7483  2.9639
##
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.04169    0.06745  -0.618    0.537
## x_s_catN     0.92968    0.06344  14.656   <2e-16 ***
## x_n_catN     0.10204    0.06654   1.533    0.126
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.055 on 242 degrees of freedom
## Multiple R-squared:  0.4753, Adjusted R-squared:  0.471
## F-statistic: 109.6 on 2 and 242 DF,  p-value: < 2.2e-16

This model correctly determines that x_n (and its one-variable model x_n_catN) do not affect the outcome. We can compare the performance of this model to the naive model on holdout data.

rmse rsquared
ypred_naive 1.303778 0.2311538
ypred_crossval 1.093955 0.4587089

The correct model has a much smaller root-mean-squared error and a much larger R-squared than the naive model when applied to new data.

## An attempted alternative: regularized models.

But cross-validation is so complicated. Can’t we just regularize? As we’ll show in the appendix of this article, for a one-variable model, L2-regularization is simply Laplace smoothing. Again, we’ll represent each “coefficient” of the one-variable model as the Laplace smoothed value minus the grand mean.

vi = ∑xj = si yi/(counti + λ) − E[yi]

Where counti is the frequency of si in the training data, and λ is the smoothing parameter (usually 1). If λ = 1 then the first term on the right is just adding one to the frequency of the level and then taking the “adjusted conditional mean” of y.

Again, let’s show this for the variable x_s.

##     x_s      sum_y count_y   grandmean         vs
## 1  s_01  20.795484      26 -0.05050187  0.8207050
## 2  s_02 -37.302227      27 -0.05050187 -1.2817205
## 3  s_03 -22.199656      28 -0.05050187 -0.7150035
## 4  s_04 -14.016649      17 -0.05050187 -0.7282009
## 5  s_05  19.622340      26 -0.05050187  0.7772552
## 6  s_06   3.129419      20 -0.05050187  0.1995218
## 7  s_07 -35.242672      30 -0.05050187 -1.0863585
## 8  s_08  36.504412      27 -0.05050187  1.3542309
## 9  s_09  33.158549      21 -0.05050187  1.5577086
## 10 s_10 -16.821957      23 -0.05050187 -0.6504130

After applying the one variable models for x_s and x_n to the data, the head of the resulting treated data looks like this:

##     x_s  x_n           y         vs         vn
## 2  s_10 n_72  0.34228110 -0.6504130 0.44853367
## 3  s_01 n_09 -0.03805102  0.8207050 0.42505898
## 4  s_03 n_18 -0.92145960 -0.7150035 0.02370493
## 9  s_08 n_43  1.77069352  1.3542309 1.28612835
## 10 s_08 n_17  0.51992928  1.3542309 0.21098803
## 11 s_01 n_78  1.04714355  0.8207050 0.61015422

Now fit the overall model:

##
## Call:
## lm(formula = y ~ vs + vn, data = dtrain_treated)
##
## Residuals:
##      Min       1Q   Median       3Q      Max
## -2.30354 -0.57688 -0.02224  0.56799  2.25723
##
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept) -0.06665    0.05637  -1.182    0.238
## vs           0.81142    0.06203  13.082  < 2e-16 ***
## vn           0.85393    0.09905   8.621  8.8e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.8819 on 242 degrees of freedom
## Multiple R-squared:  0.6334, Adjusted R-squared:  0.6304
## F-statistic: 209.1 on 2 and 242 DF,  p-value: < 2.2e-16

Again, both variables look significant. Even with regularization, the model is still overfit. Comparing the performance of the models on holdout data, you see that the regularized model does a little better than the naive model, but not as well as the correctly cross-validated model.

rmse rsquared
ypred_naive 1.303778 0.2311538
ypred_crossval 1.093955 0.4587089
ypred_reg 1.267648 0.2731756

## The Moral of the Story

Unfortunately, regularization is not enough to overcome nested model bias. Whenever you apply a y-aware process to your data, you have to use cross-validation methods (or a separate data set) at the next stage of your modeling pipeline.

### Appendix: Derivation of Laplace Smoothing as L2-Regularization

Without regularization, the optimal one-variable model for y in terms of a categorical variable with K levels {sj} is a set of K coefficients v such that

$f(\mathbf{v}) := \sum\limits_{i=1}^N (y_i - v_i)^2$

is minimized (N is the number of data points). L2-regularization adds a penalty to the magnitude of v, so that the goal is to minimize

$f(\mathbf{v}) := \sum\limits_{i=1}^N (y_i - v_i)^2 + \lambda \sum\limits_{j=1}^K {v_j}^2$

where λ is a known smoothing hyperparameter, usually set (in this case) to 1.

To minimize the above expression for a single coefficient vj, take the deriviative with respect to vj and set it to zero:

$\sum\nolimits_{x_i = s_j} -2 (y_i - v_j) + 2 \lambda v_j = 0\\ \sum\nolimits_{x_i = s_j }-y_i + \sum\nolimits_{x_i = s_j} v_j + \lambda v_j = 0\\ \sum\nolimits_{x_i = s_j }-y_i + \text{count}_j v_j + \lambda v_j = 0$

Where countj is the number of times the level sj appears in the training data. Now solve for vj:

$v_j (\text{count}_j + \lambda) = \sum\nolimits_{x_i = s_j} y_i\\ v_j = \sum\nolimits_{x_i = s_i} y_i / (\text{count}_j + \lambda)$

This is Laplace smoothing. Note that it is also the one-variable equivalent of ridge regression.

## Practical Data Science with R 2nd Edition update

We are in the last stages of proofing the galleys/typesetting of Zumel, Mount, Practical Data Science with R, 2nd Edition, Manning 2019. So this edition will definitely be out soon!

If you ever wanted to see what Nina Zumel and John Mount are like when we have the help of editors, this book is your chance!

One thing I noticed in working through the galleys: it becomes easy to see why Dr. Nina Zumel is first author.

2/3rds of the book is her work.

Posted on Tags , , Leave a comment on Free R/datascience Extract: Evaluating a Classification Model with a Spam Filter

## Free R/datascience Extract: Evaluating a Classification Model with a Spam Filter

We are excited to share a free extract of Zumel, Mount, Practical Data Science with R, 2nd Edition, Manning 2019: Evaluating a Classification Model with a Spam Filter.

This section reflects an important design decision in the book: teach model evaluation first, and as a step separate from model construction.

It is funny, but it takes some effort to teach in this way. New data scientists want to dive into the details of model construction first, and statisticians are used to getting model diagnostics as a side-effect of model fitting. However, to compare different modeling approaches one really needs good model evaluation that is independent of the model construction techniques.

This teaching style has worked very well for us both in R and in Python (it is considered one of the merits of our LinkedIn AI Academy course design):

(Note: Nina Zumel, leads on the course design, which is the heavy lifting, John Mount just got tasked to be the one delivering it.)

Zumel, Mount, Practical Data Science with R, 2nd Edition is coming out in print very soon. Here is a discount code to help you get a good deal on the book:

Take 37% off Practical Data Science with R, Second Edition by entering fcczumel3 into the discount code box at checkout at manning.com.

## vtreat Cross Validation

Nina Zumel finished new documentation on how vtreat‘s cross validation works, which I want to share here.

vtreat is a system that makes data preparation for machine learning a “one-liner” (available in R or available in Python). We have a set of starting off points here. These documents describe what vtreat does for you, you just find the one that matches your task and you should have a good start for solving data science problems in R or in Python.

The latest documentation is a bit about how vtreat works, and how to control some of the details of the work it is doing for you.

The new documentation is:

Please give one of the examples a try, and consider adding vtreat to your data science workflow.

Posted on Tags , , , Leave a comment on New vtreat Documentation (Starting with Multinomial Classification)

## New vtreat Documentation (Starting with Multinomial Classification)

Nina Zumel finished some great new documentation showing how to use Python vtreat to prepare data for multinomial classification mode. And I have finally finished porting the documentation to R vtreat. So we now have good introductions on how to use vtreat to prepare data for the common tasks of:

That is now 8 introductions to start with. To use vtreat you only have to work through one introduction (the one helping with the task you have at hand in the language you are using).

As I have said before:

• vtreat helps with project blocking issues commonly seen in real world data: missing values, re-coding categorical variables, and dealing high cardinality categorical variables.
• If you aren’t using a tool like vtreat in your data science projects: you are really missing out (and making more work for yourself).

## Why Do We Plot Predictions on the x-axis?

When studying regression models, One of the first diagnostic plots most students learn is to plot residuals versus the model’s predictions (that is, with the predictions on the x-axis). Here’s a basic example.

# build an "ideal" linear process.
set.seed(34524)
N = 100
x1 = runif(N)
x2 = runif(N)
noise = 0.25*rnorm(N)
y = x1 + x2 + noise
df = data.frame(x1=x1, x2=x2, y=y)

# Fit a linear regression model
model = lm(y~x1+x2, data=df)
summary(model)
##
## Call:
## lm(formula = y ~ x1 + x2, data = df)
##
## Residuals:
##      Min       1Q   Median       3Q      Max
## -0.73508 -0.16632  0.02228  0.19501  0.55190
##
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)
## (Intercept)  0.16706    0.07111   2.349   0.0208 *
## x1           0.90047    0.09435   9.544 1.30e-15 ***
## x2           0.81444    0.09288   8.769 6.07e-14 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.2662 on 97 degrees of freedom
## Multiple R-squared:  0.6248, Adjusted R-squared:  0.6171
## F-statistic: 80.78 on 2 and 97 DF,  p-value: < 2.2e-16
# plot it
library(ggplot2)

df$pred = predict(model, newdata=df) df$residual = with(df, y-pred)

# standard residual plot
ggplot(df, aes(x=pred, y=residual)) +
geom_point(alpha=0.5) + geom_hline(yintercept=0, color="red") +
geom_smooth(se=FALSE) +
ggtitle("Standard residual plot",
subtitle = "linear model and process")

In the above plot, we’re plotting the residuals as a function of model prediction, and comparing them to the line y = 0, using a smoothing curve through the residuals. The idea is that for a well-fit model, the smoothing curve should approximately lie on the line y = 0. This is true not only for linear models, but for any model that captures most of the explainable variance, and for which the unexplainable variance (the noise) is IID and zero mean.

If the residuals aren’t zero mean independently of the model’s predictions, then either you are missing some explanatory variables, or your model does not have the correct structure, or an appropriate inductive bias. A simple example of the second case is trying to fit a linear model to a process where the outcome is quadratically (or otherwise non-linearly) related to the outcome. To see this, let’s make an example quadratic system while deliberately failing to supply that structure to the model.

# a simple quadratic example
x3 = runif(N)
qf = data.frame(x1=x1, x2=x2, x3=x3)
qf$y = x1 + x2 + 2*x3^2 + 0.25*noise # Fit a linear regression model model2 = lm(y~x1+x2+x3, data=qf) # summary(model2) qf$pred = predict(model2, newdata=qf)
qf$residual = with(qf, y-pred) ggplot(qf, aes(x=pred, y=residual)) + geom_point(alpha=0.5) + geom_hline(yintercept=0, color="red") + geom_smooth(se=FALSE) + ggtitle("Standard residual plot", subtitle = "linear model, quadratic process") In this case, the smoothing line on the residuals doesn’t approximate the line y = 0; when the model predicts a value in the range 1 to about 2.3, it tends to be overpredicting; otherwise, it tends to underpredict. This is an instance of a pathology called “structure in the residuals.” ### The Peril of Outcomes on the x-axis What happens if you erroneously plot the residuals versus the true outcome, instead of the predictions? Let’s try this with the model for the linear process (which we know is a well-fit model): # the wrong residual graph ggplot(df, aes(x=y, y=residual)) + geom_point(alpha=0.5) + geom_hline(yintercept=0, color="red") + geom_smooth(se=FALSE) + ggtitle("Incorrect residual plot", subtitle = "linear model and process") If you make this plot when you meant to make the other, you will give yourself a nasty shock. Plotting residuals versus the outcome will always look more or less like the above graph. You might think that for a good model, the outcome and the prediction are close to each other, so the residual graphs should look about the same no matter which quantity you plot on the x-axis, right? Why do they look so different? ### Reversion to mediocrity (or the mean) One reason that the proper residual graph (for a well fit model) should smooth out to the line y=0 is known as reversion to mediocrity, or regression to the mean. Imagine that you have an ideal process that always produces a single value y. You don’t actually observe this “true value”; instead, what you observe is y plus (IID, zero mean) noise. You can build a “model” for this process that predicts the mean of the observations, in this case the value 0.1033149. Then you can calculate the residuals of your “model” in the usual way. When you plot the residuals as a function of the prediction, all the datums fall at the same horizontal coordinate of the graph, centered around zero, and approximately equally distributed between positive and negative. The “smoothing line” through this graph is simply the point (0.1033149, 0) – that is, the graph is centered at zero. On the other hand, if you plot the residuals as a function of the observed outcome, all the observations will be sorted so that the observations with positive noise are to the right of the observations with negative noise, and the smoothing line through the graph no longer looks like the line y = 0. For a process that varies as a function of the input, you can think of the prediction corresponding to an input X as the mean of all the observations corresponding to X, and the idea is the same. Incidentally, this regression to the mean is also why model predictions tend to have less range than the original training data. ### Plotting observations versus predictions Sometimes instead of plotting residuals versus the predictions, I plot observations versus predictions. In this case, you want to check that the predictions lie approximately on the line y = x. This isn’t a standard diagnostic plot, but it does give a better sense of the magnitude of the errors relative to the magnitudes of the outcomes. Again, the important thing to remember is that the predictions go on the x-axis. Here’s the correct plot: # standard prediction plot ggplot(df, aes(x=pred, y=y)) + geom_point(alpha=0.5) + geom_abline(color="red") + geom_smooth(se=FALSE) + ggtitle("Standard prediction plot") And here’s the wrong plot: # the "wrong" way ggplot(df, aes(x=y, y=pred)) + geom_point(alpha=0.5) + geom_abline(color="red") + geom_smooth(se=FALSE) + ggtitle("Incorrect prediction plot") Notice how the wrong plot again seems to show pathological structure where none exists. ### Conclusion The above examples show why you should always take care to plot your model diagnostics as functions of the predictions and not of the observations. Most students have heard this already, but we feel that demonstrating why will be more memorable that simply saying “make it so.” Posted on Tags , Leave a comment on How to Prepare Data ## How to Prepare Data Real world data can present a number of challenges to data science workflows. Even properly structured data (each interesting measurement already landed in distinct columns), can present problems, such as missing values and high cardinality categorical variables. In this note we describe some great tools for working with such data. Posted on Tags , , , Leave a comment on Preparing Data for Supervised Classification ## Preparing Data for Supervised Classification Nina Zumel has been polishing up new vtreat for Python documentation and tutorials. They are coming out so good that I find to be fair to the R community I must start to back-port this new documentation to vtreat for R. Posted on Tags , , , Leave a comment on WVPlots 1.1.2 on CRAN ## WVPlots 1.1.2 on CRAN I have put a new release of the WVPlots package up on CRAN. This release adds palette and/or color controls to most of the plotting functions in the package. WVPlots was originally a catch-all package of ggplot2 visualizations that we at Win-Vector tended to use repeatedly, and wanted to turn into “one-liners.” A consequence of this is that the older visualizations had our preferred color schemes hard-coded in. More recent additions to the package sometimes had palette or color controls, but not in a consistent way. Making color controls more consistent has been a “todo” for a while—one that I’d been putting off. A recent request from user Brice Richard (thanks Brice!) has pushed me to finally make the changes. Most visualizations in the package that color-code by group now have a palette argument that takes the name of a Brewer palette for the graph; Dark2 is usually the default. To use the ggplot2 default palette, or to set an alternative palette, such as viridis or a manually specified color scheme, set palette=NULL. Here’s some examples: library(WVPlots) library(ggplot2) mpg = ggplot2::mpg mpg$trans = gsub("\\(.*$", '', mpg$trans)

# default palette: Dark2
DoubleDensityPlot(mpg, "cty", "trans", "City driving mpg by transmission type")

# set a different Brewer color palette
DoubleDensityPlot(mpg, "cty", "trans",
"City driving mpg by transmission type",
palette = "Accent")

# set a custom palette
cmap = c("auto" = "#7b3294", "manual" = "#008837")

DoubleDensityPlot(mpg, "cty", "trans",
"City driving mpg by transmission type",
palette=NULL) +
scale_color_manual(values=cmap) +
scale_fill_manual(values=cmap)

For other plots, the user can now specify the desired color for different elements of the graph.

title = "Count of cars by number of carburetors and cylinders"

# default fill: darkblue
title = title)

# specify fill
title = title,
fillcolor = "#a6611a")

We hope that these changes make WVPlots even more useful to our users. For examples of several of the visualizations in WVPlots, see this example vignette. For the complete list of visualizations, see the reference page.

Posted on

## Advanced Data Reshaping in Python and R

This note is a simple data wrangling example worked using both the Python data_algebra package and the R cdata package. Both of these packages make data wrangling easy through he use of coordinatized data concepts (relying heavily on Codd’s “rule of access”).

The advantages of data_algebra and cdata are:

• The user specifies their desired transform declaratively by example and in data. What one does is: work an example, and then write down what you want (we have a tutorial on this here).
• The transform systems can print what a transform is going to do. This makes reasoning about data transforms much easier.
• The transforms, as they themselves are written as data, can be easily shared between systems (such as R and Python).


text:
# What is Homotopy Continuation?

We briefly explain the ideas behind HomotopyContinuation.jl

The basic idea of homotopy continuation is as follows. Suppose that you have a system of polynomials with as many variables as polynomials

$$F(x)=\begin{bmatrix} f_1(x_1,\ldots,x_n) \\ \vdots\\ f_n(x_1,\ldots,x_n) \end{bmatrix}.$$

(for more polynomials than variables, see the guide on overdetermined tracking). We wish to find a zero of $F$ in $\mathbb{C}^n$. Suppose

$$G(x)=\begin{bmatrix} g_1(x_1,\ldots,x_n) \\ \vdots\\ g_n(x_1,\ldots,x_n) \end{bmatrix}$$

is another system with a known zero: $G(\zeta) = 0$. We connect $F$ and $G$ in the space of polynomial systems by a path $t\mapsto H(x,t)$ with $H(x,0) = F(x)$ and $H(x,1)=G(x)$ (the space of polynomial systems form a vector space, in which the notion of path is well-defined).

The idea in homotopy continuation is to approximately follow the solution path $x(t)$ defined by $H(x(t),t)=0$. For this, the path is discretized into time steps $t_0 = 0 < t_1 < \cdots < t_k = 1$. If the discretization is fine enough then the Newton’s operator of $H(x,t_1)$ applied to $\zeta$ converges to a zero of $H(x,t_1)$. Then, if $t_2-t_1$ is small enough and if $\Vert \zeta_i-\xi\Vert$ is small enough for some $i\geq 0$, the iterate $\zeta_i$ converges to a zero of $H(x,t_2)$. We may repeat the procedure for $H(x,t_2)$ and starting with $\zeta_i$. Inductively, we find an approximate zero of $H(x,t_j)$ for all $j$. In the end, we will find an approximate zero for the system $F(x)=H(x,t_k)$.

Most implementations of homotopy continuation, including Bertini and HomotopyContinuation.jl, use heuristics for setting the step-sizes $t_{j+1}-t_j$ and number of Newton iterations.

For instance, the homotopy could be the straight-line homotopy $tG + (1-t)F$. The default choice of HomotopyContinuation.jl is a slightly modified straight-line homotopy. If your system has parameters, you can also make a parameter homotopy. You can even make up your own custom homotopy. For square systems is that we can automatically generate a start system $G$. If you use the basic solve command the following starting system is constructed:

$$G(x_1,\ldots,x_n) = \begin{pmatrix} x_1^{d_1} - a_1 \\ \vdots \\ x_n^{d_n} - a_n\end{pmatrix},$$

where the $a_i$ are random numbers and $d_i$ is the degree of $f_i$. There are $d_1\cdots d_n$ many solutions to this system, which are easy to write down. A theorem by Bézout says that a system whose $i$-th entry is a polynomial of degree $d_i$ has at most $d_1\cdots d_n$ solutions (if not infinitely many). Hence, tracking all $d_1 \cdots d_n$ solutions of $G$ to $F$ we are can find all solutions of $f$. Such a homotopy is called a total degree homotopy.

(For researchers interested in mathematical proofs: although the outcome of the homotopy continuation algorithms are numerical approximation of the true zeros of $F$, there is a way of rigorously certifying if a true zero is closeby or not. We refer to the paper by Hauenstein and Sottile and their algorithm alpha-Certified.)