text:
## Monday, October 29, 2007

### Compressed Sensing: Sparco helps you test all these spiffy reconstruction algorithms

With the large number of reconstruction techniques, it was just a question of time before somebody would produce a toolbox for unifying the testing of these sparse reconstruction algorithms. This is what Ewout van den Berg, Michael P. Friedlander, Gilles Hennenfent, Felix J. Herrmann, Rayan Saab, and Ozgur Yilmaz have done by releasing Sparco.

Sparco: A Testing Framework for Sparse Reconstruction

Sparco is a framework for testing and benchmarking algorithms for sparse reconstruction. It includes a large collection of sparse reconstruction problems drawn from the imaging, compressed sensing, and geophysics literature. Sparco is also a framework for implementing new test problems and can be used as a tool for reproducible research. Sparco is implemented entirely in Matlab, and is released as open-source software under the GNU Public License.
The technical report is here.

### Making sense of two findings

Two wows this past week:

First, Scott Parazynski and Daniel Tani found that there are metal shavings in one of the joint of the rotating solar arrays of the International Space Station (ISS). Since you can't change them, this is major news as it may critically impact the ISS in the future.

Second, the news that Uranium isotope ratios have been shown to not be invariant. This is the first time since the Oklo natural reactor that we show that natural reactors have occurred in many other places (though not at the rate of the Oklo). This is significant because it shows that deep burying of current nuclear waste is not unlike what is currently done in nature. Furthermore, it maybe another element showing us that Earth is a giant nuclear reactor. In this latest theory, I have always been amazed at the Origen-S and SAS2 jobs performed to compare to results found in nature. These codes are generally used in nuclear engineering to determine parameters of interest in nuclear reactor cores.

Credit: NASA, Earth-Moon system as viewed from a Mars probe on its way to Mars. 2003.

## Saturday, October 27, 2007

### Compressed Sensing: RandomFaces prove that faces lie in a low dimensional manifold

It was only a matter of time, the Machine Learning community is finding ways to use compressed sensing following the work of Richard Baraniuk and Michael Wakin on Random projections of smooth manifolds.
There were a laplacianfaces, eigenfaces, we now have randomfaces, the projection of faces onto random bases. The work of Yi Ma ([1] [2] [3])seems to show that one can get similar results in terms of classification as those obtain from the more complex techniques producing the Laplacianfaces and other Eigenfaces. Some people would say that it may parallel what is happening in the primary visual cortex.

[ Des presentations sur le compressed sensing (acquisition comprimee) se feront a l'Institut Henri Poincaré (Paris, France) en Mars 2008, Méthodes variationnelles et parcimonieuses en traitement des signaux et des images , Merci Laurent]

References:
[1] Robust Face Recognition via Sparse Representation, John Wright, Arvind Ganesh, Allen Yang and Yi Ma. Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence. [Updated January 2008]
[2] Feature Selection in Face Recognition: A Sparse Representation Perspective, Allen Yang, John Wright, Yi Ma and Shankar Sastry. Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence.
[3] Computation and Relaxation of Conditions for Equivalence between L1 and L0 Minimization, Yoav Sharon, John Wright and Yi Ma. Submitted to IEEE Transactions on Information Theory.

### Compressed Sensing: Reweighted Lp through Least square (L2), Forget Europa, let's shoot for Titan

In a previous entry, I mentioned the impressive improvement of the L1 reconstruction capability for signals that were not so sparse. We also knew that an Lp (p less than 1) exact reconstruction of sparse signals via nonconvex minimization was also possible.

Based on the work of Bhaskar Rao and Kenneth Kreutz-Delgado [2], Rick Chartrand and Wotao Yin produced an astounding reconstruction technique in this article [2]:

ITERATIVELY REWEIGHTED ALGORITHMS FOR COMPRESSIVE SENSING

The theory of compressive sensing has shown that sparse signals can be reconstructed exactly from remarkably few measurements. In [1], it was shown empirically that using Lp minimization with p less than 1 can do so with fewer measurements than with p = 1. In this paper we consider the use of iteratively reweighted algorithms for computing local minima of the nonconvex problem. In particular, a particular regularization strategy is found to greatly improve the ability of a reweighted least-squares algorithm to recover sparse signals, with exact recovery being observed for signals that are much less sparse than required by an unregularized version (such as FOCUSS, [2]). Improvements are also observed for the reweighted-l1 approach of [3].

Why is this is an important paper ? Because there is
• no need to use SDP programming, the scheme is very simple computationally. It can be written in ten lines of Matlab (Update: This algorithm was featured in this Monday Morning Algorithm)
• everybody in engineering knows least squares
• some of us now have a stab at the very large problems.

The major feature of this paper is showing that $\epsilon$ should probably be not that small in the first place. Let us hope that this heuristic will be proven in the future. In order to understand the graph above, one needs to know the acronyms:

• IRLS stands for Iteratively Reweighted or weighted Least Square (the algorithm of this paper)
• IRL1 stands for Iteratively Reweighted L1 (the case of p=0 is the same as the reweighted L1 algorithm in [3]).
K stands for the sparsity number of the signal, M is the number of measurements and N is the signal size.

Please note comparable numbers between IRL1/p=0 (linear programming algorithm in [3]) and IRLS/p=0 (least square algorithm in [1]).

Reference:
[1] ITERATIVELY REWEIGHTED ALGORITHMS FOR COMPRESSIVE SENSING, Rick Chartrand, Wotao Yin, ICASSP 2008

[2] An affine scaling methodology for best basis selection.
Rao, B.D.; Kreutz-Delgado, K. IEEE Transactions on Signal Processing, vol.47, (no.1), IEEE, Jan. 1999. p.187-200. [Rao1999.pdf]

[3] Enhancing sparsity by reweighted L1 minimization, Emmanuel Candès, Mike Wakin and Stephen Boyd

### Compressive Sensing Radiation Detector Part II

In the case where radiation does not suffer much scattering, the principle of Coded Aperture used for light can be directly applied to these radiation fields. Berthold Horn, Richard Lanza, Roberto Accorsi,
Klaus Ziock, and Lorenzo Fabris at MIT have been following this principle to do imaging by performing Computational Imaging. As pointed out before, Coded Aperture is also what the DISP group does at Duke University (but with radiation that are in the visible range). The big difference resides in the Mask, in radiation with smaller wavelengths (X-rays,...), radiation goes through the mask, but more generally you cannot guide the radiation to your detector and so it becomes a little bit more complex. Spatial resolution is extremely difficult to do because the detector are expensive and need to be where the radiation goes. So one has to resort to batteries of detectors such as the ones below.
In order to provide spatial resolution, one has to either to have many detectors or move the detector or the source as can be seen in this animation (“Synthetic Aperture” radiography). An introduction to instrumentation using coded aperture for astronomy can be found here. This is a great introduction to the subject, especially in regards to the current inversion techniques.

When one deals with radiation scattering (neutrons in nuclear reactor cores or infra-red/visible light in tissues) , radiation sensors have a much more difficult kernel to invert (because the diffusion/scattering of the rays in matter) and requires methods like Bayesian Compressed Sensing (i.e. In-situ detector of Lawrence Carin, Dehong Liu, and Ya Xue at Duke University). All in all, it looks like some of these techniques for Coded Aperture could be improved at the software and hardware level using Compressed Sensing.

## Wednesday, October 24, 2007

### New satellite imagery of the California fires

ESA just released new satellite imagery from the California fires using the MERIS camera on ENVISAT. These shots were taken 3 hours ago. IMagery from two days ago can be found in a previous entry.

I made a close up version centered on L.A.

The full imagery is shown below. The cloud at the bottom is clearly different in nature than the ones at the top. i.e. they are the result of the fire.

### California's fires from Space using different satellites

[ Update: new imagery, October 24th can be found here]

NASA has been able to provide imagery with the MODIS rapid response system where they seem to be using data from Aqua and Terra (which I believe are on the same orbit as Landsat.) I did a search on the USGS database, and found no Landsat 5 (it has had some on-board problems since October 6th) and only one Landsat 7 image. The Landsat 7 one is not clear as it has the SLC off and therefore looks pretty bad.

I have found a very nice image taken on October 22nd by the MERIS camera on-board the European ENVISAT. There should be a new one in about 4 hours.

I am a little bit disturbed that there isn't only one UI where I can give a location of interest, a time line and find out if either the USGS, NASA and ESA or other civilian satellites can tell me if there are any data associated with this criteria. It was a pain when doing this type of search when looking for clues on Jim Gray's boat, it continues to be a pain to this day.

Credit Image: ESA/Envisat

### Trace: What is it good for ? - How Compressed Sensing relates to Dimensionality Reduction

In reference [1] by Killian Weinberger and Lawrence Saul, one learns that one dimensionality reduction technique uses the maximization of the trace of the Gram matrix in order to unfold a manifold . From [1]

..We also emphasize that in eq. (9), we are maximizing the trace, not minimizing it. While a standard relaxation to minimizing the rank [9] of a semidefinite matrix is to minimize its trace, the intuition here is just the opposite: we will obtain a low dimensional embedding by maximizing the trace of the Gram matrix...

At the same time, Benjamin Recht, Maryam Fazel, and Pablo Parrilo provide a somewhat different view in [3] (Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization a paper that draws a parallel between Compressed Sensing using the L1 norm and the nuclear norm for finding low rank approximation of a matrix ) where the Trace of a positive definite matrix (the Gram Matrix) is minimized. The reason is simple [3]:

The nuclear norm is a convex function, can be optimized efficiently, and is the best convex approximation of the rank function over the unit ball of matrices with norm less than one. When the matrix variable is symmetric and positive semidefinite, this heuristic is equivalent to the trace heuristic often used by the control community.....
...
Low-dimensional Euclidean embedding problems
A problem that arises in a variety of fields the determination of configurations of points in low-dimensional Euclidean spaces, subject to some given distance information. In Multi-Dimensional Scaling (MDS) such problems occur in extracting the underlying geometric structure of distance data.
So in one series of paper, we have a minimization and in the other one we have a maximization of the trace of a Gram matrix as an objective function in order to obtain a reduced dimension/low rank approximation of that matrix. This seems odd.

When one goes back to one of the first paper of Killian Weinberger and Lawrence Saul [1], one can read how the heuristic trace argument comes about:

The constraints for local isometry under this neighborhood relation are simply to preserve the lengths of the edges in this graph, as well as the angles between edges at the same node. In practice, it is easier to deal only with constraints on distances, as opposed to angles.....Thus, we must further constrain the optimization to the cone of semidefinite matrices [21]. In sum, there are three types of constraints on the Gram matrix Kij , arising from local isometry, centering, and semidefiniteness. The first two involve linear equality constraints; the last one is not linear, but importantly it is convex. We will exploit this property in what follows.What function of the Gram matrix can we optimize to “unfold” a manifold, as in Fig. 1? As motivation, consider the ends of a piece of string, or the corners of a flag. Any slack in the string serves to decrease the (Euclidean) distance between its two ends; likewise, any furling of the flag serves to bring its corners closer together. More generally, we observe that any “fold” between two points on a manifold serves to decrease the Euclidean distance between the points. This suggests an optimization that we can perform to compute the outputs Yi that unfold a manifold sampled by inputs Xi. In particular,we propose to maximize the sum of pairwise squared distances between outputs....By maximizing eq. (6), we pull the outputs as far apart as possible, subject to the constraints in the previous section. Before expressing this objective function in terms of the Gram matrix Kij , let us verify that it is indeed bounded, meaning that we cannot pull the outputs infinitely far apart. Intuitively, the constraints to preserve local distances (and the assumption that the graph is connected) prevent such a divergence.... Thus, the objective function cannot increase without bound if we enforce the constraints to preserve local distances.

We can express the objective function in eq. (6) directly in terms of the Gram matrix Kij of the outputs Yi. Expanding the terms on the right hand side, and enforcing the constraint that the outputs are centered on the origin, we obtain: T (Y ) =Tr(K). (9)
Thus, we can interpret the objective function for the outputs in several ways: as a sum over pairwise distances in eq. (6), as a measure of variance in eq. (9), or as the trace of their Gram matrix in eq. (9).

In other words, using the max trace argument one finds a lower dimensional manifold by unfolding that manifold subject to distance constraints. In the same way, an argument based on the minimization of the trace could be depicted as squashing that manifold subject to similar distance or angle constraints. Either way, one obtains a reduced version of the manifold : in the MVU case (max trace), some respect is given to the geodesic distance on the embedding whereas in the min case, respect is given to the Euclidian distance. Obviously, in dimension 3, one wonders how a Mobius strip would fare in either situations. This is my interpretation so far, if you have better insight, feel free to enlighten me on this issue. As suggested by a commenter, I just re-read Jon Dattorro's very nice book and it looks like this description of the minimization business is similar to his.( see footnote 4.12, page 241):

Trace (trG = hI , Gi) minimization is a heuristic for rank minimization. (§7.2.2.1) It may be interpreted as squashing G which is bounded below by X^T X as in (512).

References:

[1] Unsupervised learning of image manifolds by semidefinite programming, K.Q. Weinberger and L. K. Saul (2004). (Outstanding student paper award). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR-04), Washington D.C. pdf

[2] Graph Laplacian Regularization for Large-Scale Semidefinite Programming, K. Q. Weinberger, F. Sha, Q. Zhu and L. K. Saul (2007). In B. Schoelkopf, J. Platt, and T. Hofmann (eds.), Advances in Neural Information Processing Systems 19. MIT Press: Cambridge, MA. pdf

[3] Benjamin Recht, Maryam Fazel, and Pablo Parrilo, Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization

[4] Jon Dattorro, Convex Optimization & Euclidean Distance Geometry, Chapter 4.

## Tuesday, October 23, 2007

### Compressed Sensing: Author Needs Help

Does any reader of this blog have the following paper in their harddrive ?

David Donoho and Yaakov Tsaig, Fast solution of ell-1-norm minimization problems when the solution may be sparse. (Preprint, 2006)

It used to be at: http://www.stanford.edu/%7Etsaig/Papers/FastL1.pdf

The html version is available from Google. It was listed on the Compressed Sensing repository website as of January 6, 2007 and was accessible until recently. The Compressed Sensing repository at Rice did not have a local copy and one of the author is looking for your help in finding this preprint (his account was wiped out).

## Thursday, October 18, 2007

### Compressed Sensing: Hardware Implementation, Part III, The explosive birth of coded aperture.

{Update Nov. '08, I have made a list of most Compressed Sensing Hardware here ]

Before we start on the subject of hardware implementation, there is a new and comprehensive tutorial on Compressed Sensing by Mike Wakin and Justin Romberg.

I initially started the listing of hardware implementation in compressed sensing though the presentation of the single pixel camera at Rice, the random lens imager at MIT, and some spectrometer work at Duke, then there was a hyperspectral imager at Yale and an analog imager at Georgia Tech. I have seldom mentioned the obvious hardware implementation of the whole MRI work spearheaded at Stanford but this one doesn't require hardware modification per se, rather a change of operation of the hardware. There is also the whole work surrounding the design of new A/D hardware named A/I (analog to information) that will be useful for capturing very high frequency signals either through a Nonuniform sampler (NUS) or a Random pre-integrator (RPI) .

More work has been performed on some of these concepts by looking at how one can use the sparsity of spectral measurement and spatial information. At MIT, I had mentioned the Random Lens Imager ( Random Lens Imaging, Rob Fergus, Antonio Torralba, Bill Freeman ). In that project, they mentioned the fact that using machine learning techniques one could provide depth information from single shots. Instead of doing that, they have resorted to putting additional filters in a lens to eventually provide depth information:

Image and Depth from a Conventional Camera with a Coded Aperture by Anat Levin, Rob Fergus, Fredo Durand, and Bill Freeman
A conventional camera captures blurred versions of scene information away from the plane of focus. Camera systems have been proposed that allow for recording all-focus images, or for extracting depth, but to record both simultaneously has required more extensive hardware and reduced spatial resolution. We propose a simple modi_cation to a conventional camera that allows for the simultaneous recovery of both (a) high resolution image information and (b) depth information adequate for semi-automatic extraction of a layered depth representation of the image. Our modification is to insert a patterned occluder within the aperture of the camera lens, creating a coded aperture. We introduce a criterion for depth discriminability which we use to design the preferred aperture pattern. Using a statistical model of images, we can recover both depth information and an all-focus image from single photographs taken with the modified camera. A layered depth map is then extracted, requiring user-drawn strokes to clarify layer assignments in some cases. The resulting sharp image and layered depth map can be combined for various photographic applications, including automatic scene segmentation, post-exposure refocussing, or re-rendering of the scene from an alternate viewpoint.
One can see the similarity with the work performed by Andrew Ng and Ashutosh Saxena on learning depth from monocular images where a MAP estimation using Laplacian (sparse) priors is performed to infer depth. The point is that this technique can be readily using reconstruction techniques of compressed sensing in order to reconstruct the data.

Also using coded aperture and mentioned in another post is the work performed at Duke on spectrometers and hyperspectral imagers. The paper of Ashwin Wagadarikar, Renu John, Rebecca Willett, David Brady, entitled "Single disperser design for compressive, single-snapshot spectral imaging," in Adaptive Coded Aperture Imaging and Non-imaging Sensors, D. Casasent and T. Clark, eds., Proc. SPIE 6714 (2007) is still not available. However we know that the reconstruction of the spectral cube is performed using Gradient Projection for Sparse Reconstruction (GPSR) method (please note there is new version: GPSR 3.0) . While it is not yet released, Scott (a commenter) pointed out that there is another paper out on a similar subject: Michael Gehm, Renu John, David Brady, Rebecca Willett, and Tim Schulz, "Single-shot compressive spectral imaging with a dual-disperser architecture," Opt. Express 15, 14013-14027 (2007)
This paper describes a single-shot spectral imaging approach based on the concept of compressive sensing. The primary features of the system design are two dispersive elements, arranged in opposition and surrounding a binary-valued aperture code. In contrast to thin-film approaches to spectral filtering, this structure results in easily-controllable, spatially-varying, spectral filter functions with narrow features. Measurement of the input scene through these filters is equivalent to projective measurement in the spectral domain, and hence can be treated with the compressive sensing frameworks recently developed by a number of groups. We present a reconstruction framework and demonstrate its application to experimental data.

Where the solution is reconstructed using an Expectation-Maximization algorithm (e.g. a regularized version of the Richardson-Lucy algorithm). Other related information can be found in references [1] [2] [3].

With Hyper-GeoCam, a random lens imager based on the MIT design, we used random elements that would diffract wavelengths in different spatial locations, thereby enabling a random spectral-spatial mixing akin to the Duke design. The unmixing of these components is therefore directly amenable to Compressed Sensing reconstruction techniques. The main difference between those carefully crafted coded aperture imagers (Duke) and that of the random lens imager reside in the complexity of the reconstruction technique. The reason the random lens imager is not "taking off"(ahah) has to do with the fact that, in the calibration process, one cannot perform a matrix reconstruction with 10^6 elements even tough most of them are sparse. The current techniques just do not allow for it yet (they do not scale well above some hundreds constraints/variables). The crafted coded aperture imager provides an idea of where those terms are hence facilitating the location of the non-zero component of the calibration matrix. Another aspect of the problem is that with random materials, reflection may be wavelength sensitive and the whole calibration process is instrument specific. I am wondering aloud if the technique on Graph Laplacian Regularization for Large-Scale Semidefinite Programming [6] would not much help with respect to the calibration of the random lens imager.

References of interest:
[1] David Brady's lectures on computational imaging
[2] Coded aperture spectroscopy and spectral tomography
[3] Optical Imaging
[4] Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady, "Single disperser design for coded aperture snapshot spectral imaging," Submitted to a feature issue on Computational Optical Sensing and Imaging (COSI), Applied Optics (2007)
[5]Ashwin Wagadarikar, Michael Gehm, and David Brady, "Performance comparison of aperture codes for multimodal, multiplex spectroscopy," Applied Optics 46, 4932-4942 (2007)
[6] Graph Laplacian Regularization for Large-Scale Semidefinite Programming, K. Q. Weinberger, F. Sha, Q. Zhu and L. K. Saul (2007). In B. Schoelkopf, J. Platt, and T. Hofmann (eds.), Advances in Neural Information Processing Systems 19. MIT Press: Cambridge, MA

## Tuesday, October 16, 2007

### Judging the Autism Charts Challenge

When writing this entry on the difficulty of displaying data when diagnosing Autism at age 2, I did not expect such a large number of people putting their minds into making the graphs better. Andrew Gelman and Masanao Yajima made a challenge out of it. It got picked up by the folks at JunkCharts, EagerEyes.org, and at Perceptual edge and much brain computing cycle went into this Challenge. Previously, I had already stated that I thought Autism was a Grand Challenge and this is in no small part because the diagnosis phase is not early enough, not accurate enough and sometimes not useful enough.
• Not early enough is the reason why this study [1] by Catherine Lord was performed. Instead of taking a "normal" test at age 3, age 2 is preferred with regards to intervention.
• Not accurate enough and not useful: there is currently very little difference whatsoever in treatment between kids that have been diagnosed Autistic or with Pervasive Developmental Disorder- Not Other Specified (PDD-NOS).
Eventually, one cannot even devise clinical trials for drug development until some of these tests are better refined. The figures entered as part of the Challenge can be seen in full scale here:

Please go to Andrew Gelman and Masanao Yajima's blog and answer the survey at the very end. Please note there are TWO questions.
• Which plot did you like best?
• Which plots did you find useful?
Please answer them both.. Thank you to Antony Unwin, Stack Lee, Robert Kosara, Patrick Murphy, Andrew Gelman and Masanao Yajima for contributing to this effort.

[1] Autism From 2 to 9 Years of Age, Catherine Lord, Susan Risi, Pamela S. DiLavore, Cory Shulman, Audrey Thurm, Andrew Pickles. Arch Gen Psychiatry. 694-701, Vol. 63, June 2006

Liked this entry ? subscribe to the Nuit Blanche feed, there's more where that came from

## Friday, October 12, 2007

It's funny, it looks like one of the ways I am staying on top of several subjects that are not part of my day-to-day job resemble some of the techniques Jessica Logan describes as potentially interesting for memorization. She just gave a seminar at Rice on "
Brain and Behavior: How We Create and Maintain Memory in Early Adulthood and Advanced Age” - No one likes it, but it happens to the best of us—that frustrating feeling of trying to remember something you know that you knew at one time, but which now inexplicably eludes you. From older adults lamenting another "senior moment" to students stumped on a final exam, memory failures are an unwelcome but woefully familiar experience for everyone. This talk will focus on research that strives to help us understand how some of these failures occur and how we can improve our chances of avoiding them in the future. Cognitive research that integrates both behavioral and neuroimaging (fMRI) techniques to explore memory formation and retrieval in healthy younger and older adults will be discussed.

In her fascinating presentation, she makes the point that studies seem to show that cramming for a subject area is worse in the long run with regards to remembering it. She also present results for two other techniques called equal and spaced interval technique. In cramming, the memorization happens by repeating over and over the same subject area and then take a test on it. Most college students know that the technique might work for a test but that they are not optimal strategies for the long term. The two other techniques are using some type of time interval between repeating the same subject area in order to reinforce memorization. The equal time interval asks for an equal amount of time between recalls whereas the spaced interval techniques demands that recall occur at different (random) time intervals. Interestingly, young adult fare badly in recalling memory acquired through the spaced and timed intervals whereas old adults do see a benefit to these methods (as opposed to cramming). Additional information can be found in this paper [1]. I find it fascinating because the way this blog is structured and how I set up my desk remind me on how I eventually always randomly come back to a subject, rediscover it and remind myself very quickly of the important lessons from it. The market for product on memory is very large, yet none of these techniques seem to be proposed. I note her statement about spaced memorization:
• Making practice more difficult initially can produce better performance on a final test
• Challenging the learner in between practice attempts enhances the benefit of repetition
I think this is the main reason why people should read this blog on compressed sensing when trying to learn something else. The issue of sleep is also mentioned indirectly in her 24 hours study and I wonder if they yield the same results as for the sensorimotor control.

Reference: [1] Does Expanded Retrieval Produce Benefits Over Equal-Interval Spacing? David Balota, Janet Duchek, Susan Sergent-Marshall, and Henry Roediger III, Department of Psychology, Washington University.
Photo credit: ESA/NASA/ASI/JPL/SSI, Radar view of Titan's methane lakes.

## Thursday, October 11, 2007

### Producing Maps using Commercial Overflights: Part deux

In a previous entry, I mentioned the simple possibility of producing maps from commercial airliners using point and shoot cameras. The reason for that post was to show real examples that can be obtained from different heights and how these images can be put together using state of the art stitching programs. In the comment section Paul asked the questions:

the only questions I have are whether a) they'd be systematic enough to provide reliable coverage, and b) who'd be tasked with cleaning and analyzing the images (always a pain in the whatnot).

Given that most commercial flights go along similar flight paths, would the coverage be potentially comprehensive enough to be useful?

What area magnitude could be covered by a single shot of the sensor at 35000 feet?

Let me first try to address the issue of coverage which seems to be an underlying issue. First let us remember that this solution is really when you have to survey or map a region hours if not minutes after a disaster. We are talking about finding out if specific large structures, roads, villages, town are in good shape or not. This can be applied to either New Orleans and all its surrounding regions right after Katrina or to California if the big one hits a large swath of land. In both cases, we know that satellites will provide imagery within the next three to six days. In some other countries where accurate maps do not exist, we may not have meaningful satellite coverage at all. In either of these extreme cases, we are talking about providing as much information as possible. The coverage of this technique might be small compared to Google Maps or Google Earth or to a specialized satellite campaign but the issue is trumped in that the information is timely. Obviously there is the issue of flight corridors but I am willing to bet that a Houston-Miami flight could happen to take a another lawful corridor if somehow the pilot knew that some imagery made in the plane could help some of the search and rescue effort in New Orleans. Let us also note that private efforts could yield improved coverage. If one follows the example of the NOAA that flew Cessnas over the Katrina ravaged area and incorporated their results into Google Maps. While NOAA is a government agency, a similar private effort could be undertaken. Please note the overlapping photos in these NOAA shots: stitching these images is easy to do with the software I mention later. The difference between this NOAA effort and one using commercial overflight with point and shoot cameras reside in three ways
• no need to retrofit a plane with an outside camera
• no need for GPS correction, pointing hardware and engineer's time, the stitching algorithm does the map
• it has the ability to provide much faster response time.

The other enabling feature of this idea can be tracked back to the recent advances made in the intelligent stitching algorithm borne out of machine vision. Currently, the algorithm can automatically "merge" photos taken from nearly the same altitude, point of view and optical focus by different cameras. The merging/stitching is done absolutely automatically: No need for a human expert in the loop when putting those images together. You should not trust me, you try it. The free version of autopano produces panorama with watermarks. So next time you take a plane, use your point and shoot camera and try it.

Why would we need several shots of the same place on different airplanes or from different people in the same plane ? Mostly because of the clouds. Clouds are a pain and one can begin to remove them only if you have enough pictures that eventually cover the ground. Also, since window seats are on one side, it pays to have people from either side point their cameras in different directions. It has been my experience that in airliners, the best window was the one next to the flight attendants at the very end of the plane. It also allow you to have a view near straight down (nadir). I'll make a more technical note on what worked and what did not in a future entry.

With regard to actual coverage at 35,000 feet, one can begin to have an idea of the features of interest by usign Google Earth and get to a 35,000 feet altitude. Generally, the camera has a smaller view angle so one really sees less than that in the plane. But I would venture, that we can get a swath of at least 30 miles of useful data (depending on the optical zoom of the point and shoot camera).

To summarize, you get what you can with regards to coverage, but the more cameras in the air the most likely you get data that accumulates to eventually produce some type of a map. No need for experts to clean up the data, just a laptop, a stitching software and a point and shoot camera will do.

Eventually the idea is pretty simple, once a disaster area is known and that commercial airlines are flying over it, willing passengers take photos during the whole trip over from their window seat (with some simple instructions). Once landed either they give their 8GB sd cards to somebody, upload their data directly on the laptop at the airport arrival or upload them on the web. Then, using one of these intelligent stitcher program that cost \$100 you can put all the images together even if they have been taken by different cameras at different times (given the altitude was about the same). One can then produce a relatively large map of land and put it on the web without the need to "connect" it to google maps or google earth.

Photo 1: This picture was taken at 35,000 feet over Iceland. Another plane is using the same corridor ?

Photo 2: Credit NOAA

## Tuesday, October 09, 2007

### Compressed Sensing: Reweighted L1 and a nice summary on Compressed Sampling

In a previous entry, I noted that Emmanuel Candès had presented a new iterative scheme at IMA that was sure to improve the current L1 reconstruction techniques used in Compressed Sensing. The scheme was called reweighted L1. It is now the subject of a technical report: Enhancing sparsity by reweighted L1 minimization by Emmanuel Candès, Mike Wakin and Stephen Boyd

It details the technique but also features a discussion on the reconstruction with overcomplete dictionaries, a comparison with the l2 based FOCUSS algorithm and a brief comparison with Chartrand's work. I noted two items from this paper which I am sure will be developed later:

• The potential for the analysis-based reconstruction being superior to the synthesis based l1 recovery (they were shown to be different in the case of overcomplete bases see [1],[2])
• The potential usefulness of nonconvex penalties in sparse signal recovery

It also looks like there is an issue coming up of IEEE Signal Processing Magazine on Compressed Sensing in which Candes and Wakin have written the following: People hearing without listening: an introduction to compressive sampling.

This is a well written introduction to Compressed Sensing with a particular attention to explaining clearly the Restricted Isometric Property. There is also a good summary of the Analog to Information (A/I) work with a mention on how reweighted L1 being used in that context. Much of it had already been shown on the IMA videos, now it is on paper. The most fascinating part of the A/I attempts is the reason why Compressed Sensing is really needed: the ability to sample at a much higher rate than currently feasible.
Whereas it may not be possible to digitize an analog signal at a very high rate rate, it may be quite possible to change its polarity at a high rate. The idea is then to multiply the signal by a pseudo-random sequence of plus and minus ones, integrate the product over time windows, and digitize the integral at the end of each time interval. This is a parallel architecture and one has several of these random multiplier-integrator pairs running in parallel using distinct or event nearly independent pseudo-random sign sequences.
When reading this, I could not but stop thinking how it could be used in optical interferometry.

References:
[1] M. Elad, P. Milanfar, and R. Rubinstein, "Analysis Versus Synthesis in Signal Priors", Inverse Problems. Vol. 23, no. 3, pages 947-968, June 2007.

[2] J.-L. Starck, M. Elad, and D.L. Donoho, "Redundant Multiscale Transforms and their Application for Morphological Component Analysis", the Journal of Advances in Imaging and Electron Physics, Vol. 132, pp. 287-348, 2004.

## Friday, October 05, 2007

### Compressed Sensing: Chaining Pursuit Code for dimension reduction in the L1 norm for sparse vectors

Make that seven codes useful to do compressed sensing. Joel Tropp made available a Matlab toolbox to perform Compressed Sensing using Chaining pursuit [tar.gz]. The Chaining Pursuit algorithm was explained in the following paper entitled Algorithmic linear dimension reduction in the l1 norm for sparse vectors by Anna Gilbert, Martin Strauss, Joel Tropp, and Roman Vershynin:

Using a number of different algorithms, we can recover approximately a sparse signal with limited noise, i.e, a vector of length d with at least d−m zeros or near-zeros, using little more than mlog(d) nonadaptive linear measurements rather than the d measurements needed to recover an arbitrary signal of length d. We focus on two important properties of such algorithms.
• Uniformity. A single measurement matrix should work simultaneously for all signals.
• Computational Efficiency. The time to recover such an m sparse signal should be close to the obvious lower bound, mlog(d/m).
This paper develops a new method for recovering m sparse signals that is simultaneously uniform and quick. We present a reconstruction algorithm whose run time, O(mlog2(m) log2(d)), is sublinear in the length d of the signal. The reconstruction error is within a logarithmic factor (in m) of the optimal m-term approximation error in l1. In particular, the algorithm recovers m-sparse signals perfectly and noisy signals are recovered with polylogarithmic distortion. Our algorithm makes O(mlog2(d)) measurements, which is within a logarithmic factor of optimal. We also present a small space implementation of the algorithm.
These sketching techniques and the corresponding reconstruction algorithms provide an algorithmic dimension reduction in the l`1 norm. In particular, vectors of support m in dimension d can be linearly embedded into O(mlog2 d) dimensions with polylogarithmic distortion. We can reconstruct a vector from its low-dimensional sketch in time O(mlog2(m) log2(d)). Furthermore, this reconstruction is stable and robust under small perturbations.

Joel also came out with bounds of interest to the previous small example I gave earlier on approximating sines with diracs (Compressed Sensing: How to Wow your friends). The article is entitled: "On the linear independence of spikes and sines" [ .pdf | arXiv math.FA 0709.0517 ]

Photo Credit: NASA/Johns Hopkins University Applied Physics Laboratory/Southwest Research Institute: New Horizons Spacecraft watching Io and its volcano and Europa in the same photo shoot.

format: plain


text:
Chapter Test
Chapter
Chapter 5
Section
Chapter Test
Solutions 10 Videos

Match

A  \displaystyle y =\frac{5x + 2}{x -1}  and B  \displaystyle y=\frac{1}{2x -1}  with the graph below.

Q1

Suppose that n is a constant and that f(x) is a linear or quadratic function defined when x = n. Complete the following sentences.

If f(n) is large, then \frac{1}{f(n)} is ...

Q2a

Suppose that n is a constant and that f(x) is a linear or quadratic function defined when x = n. Complete the following sentences.

If f(n) is small, then \frac{1}{f(n)} is ...

Q2b

Suppose that n is a constant and that f(x) is a linear or quadratic function defined when x = n. Complete the following sentences.

If f(n) = 0, then \frac{1}{f(n)} is ...

Q2c

Suppose that n is a constant and that f(x) is a linear or quadratic function defined when x = n. Complete the following sentences.

If f(n) = 0 is positive, then \frac{1}{f(n)} is ...

Q2d

Without using graphing technology, sketch the graph of  \displaystyle y =\frac{2x + 6}{x -2}

Q3

A company purchases x kilograms of steel for \$2249.52. The company processes the steel and turns it into parts that can be used in other factories. After this process, the total mass of the steel has dropped by 25 kg (due to trimmings, scrap, and so on), but the value of the steel has increased to \$ 10, 838.52. The company has made a profit of \\$2/kg. What was the original mass of the steel? What its the original cost per kilogram?

Q4

Select a strategy to solve each of the following.

 \displaystyle \frac{-x}{x -1}= \frac{-3}{x + 7}

Q5a

Select a strategy to solve each of the following.

 \displaystyle \frac{2}{x + 5} > \frac{3x}{x + 10}

 \displaystyle f(x) =\frac{ax+ b}{cx + d} , explain


text:
Categories:

Numeric Functions

# WIDTH_BUCKET¶

Constructs equi-width histograms, in which the histogram range is divided into intervals of identical size, and returns the bucket number into which the value of an expression falls, after it has been evaluated. The function returns an integer value or null (if any input is null).

## Syntax¶

WIDTH_BUCKET( <expr> , <min_value> , <max_value> , <num_buckets> )

## Arguments¶

expr

The expression for which the histogram is created. This expression must evaluate to a numeric value or to a value that can be implicitly converted to a numeric value.

The value must be within the range of -(2^53 - 1) to 2^53 - 1 (inclusive).

min_value and max_value

The low and high end points of the acceptable range for the expression. The end points must also evaluate to numeric values and not be equal.

The low and high end points must be within the range of -(2^53 - 1) to 2^53 - 1 (inclusive). In addition, the difference between these points must be less than 2^53 (i.e. abs(max_value - min_value) < 2^53).

num_buckets

The desired number of buckets; must be a positive integer value. A value from the expression is assigned to each bucket, and the function then returns the corresponding bucket number.

When an expression falls outside the range, the function returns:

• 0 if the expression is less than min_value.

• num_buckets + 1 if the expression is greater than or equal to max_value.

## Example¶

Create a four-bucket histogram on the price column for homes sold in the price range of \$200 - 600k, ordered by sales date. The function returns the bucket number (SALES GROUP) for each value in the set.

Create and fill a table:

CREATE TABLE home_sales (
sale_date DATE,
price NUMBER(11, 2)
);
INSERT INTO home_sales (sale_date, price) VALUES
('2013-08-01'::DATE, 290000.00),
('2014-02-01'::DATE, 320000.00),
('2015-04-01'::DATE, 399999.99),
('2016-04-01'::DATE, 400000.00),
('2017-04-01'::DATE, 470000.00),
('2018-04-01'::DATE, 510000.00);

Query the table, calling WIDTH_BUCKET():

SELECT
sale_date,
price,
WIDTH_BUCKET(price, 200000, 600000, 4) AS "SALES GROUP"
FROM home_sales
ORDER BY sale_date;
+------------+-----------+-------------+
| SALE_DATE  |     PRICE | SALES GROUP |
|------------+-----------+-------------|
| 2013-08-01 | 290000.00 |           1 |
| 2014-02-01 | 320000.00 |           2 |
| 2015-04-01 | 399999.99 |           2 |
| 2016-04-01 | 400000.00 |           3 |
| 2017-04-01 | 470000.00 |           3 |
| 2018-04-01 | 510000.00 |           4 |
+------------+-----------+-------------+


text:
## A community for students. Sign up today

Here's the question you clicked on:

## anonymous 3 years ago Find the most general antiderivative of the function. g(t) = (5 + t + t^2)/ square root of t

• This Question is Closed
1. myininaya

$g(t)=\frac{5+t+t^2}{t^\frac{1}{2}}=\frac{5}{t^\frac{1}{2}}+\frac{t}{t^\frac{1}{2}}+\frac{t^2}{t^\frac{1}{2}}=5t^{-\frac{1}{2}}+t^{1-\frac{1}{2}}+t^{2-\frac{1}{2}}$ $g(t)=5t^{-\frac{1}{2}}+t^\frac{1}{2}+t^{\frac{3}{2}}$ Find the antiderivative of each term using $f(t)=t^n => F(t)=\frac{t^{n+1}}{n+1} \text{ where } F'=f \text{ and } n \neq -1$

2. anonymous

thanks!

#### Ask your own question

Sign Up
Find more explanations on OpenStudy
Privacy Policy


text:
# Talk:Gumbel distribution

WikiProject Statistics (Rated Start-class, Low-importance)

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

Start  This article has been rated as Start-Class on the quality scale.
Low  This article has been rated as Low-importance on the importance scale.

I agree; maybe I'll get to it at some point. I've done some small fixes, but more is needed. Michael Hardy 22:11, 23 Feb 2004 (UTC)

If I may give a suggestion, I would redirect the page to the on the generalized extreme value distribution (which I just wrote and I hope in a decent way) of which the Fisher-Tippett is a special case. Since I'm a newbie, I'm not sure it is appropriate to do so, so I prefer to ask.

At any rate, the total lack of any references is troubling, and the fact that neither Pi nor the Euler constant are expressly defined is sloppy. -- Anon.

## Symbol mismatch - text and graphics

The images use symbols β and γ, but the text uses β and μ. One or the other needs correcting for consistency. DFH 20:25, 30 January 2007 (UTC)

I suppose those symbols are BETA, GAMMA, and MU. In the table at right MU is the location parameter which is not the mean.
The two top right graphs use parameters LAMBDA and BETA but the table below them uses MU and BETA. Is LAMBDA also the location parameter?
Is consistency across articles on probability distributions intended regarding symbols for location, scale, and shape parameters? If not entirely, is consistency intended for the top right graphs? for the righthand tables? —Preceding unsigned comment added by 140.247.23.33 (talk) 23:56, 14 November 2007 (UTC)

changed the figures to remove the figure inconsistencies. --Herr blaschke (talk) 11:12, 26 March 2008 (UTC)

## Distribution name

Isn't this also called the Gompertz distribution? If you see, the Shifted Gompertz distribution links this distribution, but nothing is written here about the other name, so it is confusing. Faermi 16:03, 16 May 2007 (UTC)

## Mean Value Calculation

I think that tere is a mistake in the calculation of the mean value, because I've done some experiments in Mattab and the sign in the equation should be (-), i.e., mean$=\mu - \beta\,\gamma\!$, and not mean$=\mu + \beta\,\gamma\!$, as it was before.

Anyone can confirm if that's right?

Thank you all.
08:37, 3 April 2008 (UTC)

Response: see Location-scale_family for the rules of location scale transormations. Also, it is cited with a + in the body of the article. If you are careful with the - signs in the distribution, you will see that the + sign in the mean is the correct formula. —Preceding unsigned comment added by 131.110.112.62 (talk) 20:17, 4 April 2008 (UTC)

Response 2: Matlab has a different definition of the Gumbel distribution from that used here (it has a heavy left tail). Thus the mean of the Matlab Gumbel distribution is indeed $=\mu - \beta\,\gamma\!$. This caused someone to incorrectly change the sign to minus on 27 Nov. 2010. I have now changed it back to plus. Kristjan.Jonasson (talk) 11:55, 30 December 2010 (UTC)

## Gumbel graph paper

The attached image of the graph paper doesn't make much sense. How is it related to this distribution? This should be explained. --Ghtx (talk) 15:27, 5 May 2009 (UTC)

i think something is wrong. the maximum is not the first order statistic? which ( the first order statistic or the nth order statistic) is distributed as gumbel? —Preceding unsigned comment added by 41.91.62.24 (talk) 15:06, 25 November 2010 (UTC)

## Minimum vs Maximum Case

From the text: "It is also known as the log-Weibull distribution"

Actually, per NIST, the Gumbel (minimum case) distribution is the distribution of the log of a Weibull-distributed variable, whereas this page is talking about the Gumbel (maximum case) distribution... — Preceding unsigned comment added by 2001:4898:0:FFF:0:5EFE:A7D:8603 (talk) 21:32, 26 October 2012 (UTC)


text:
# Problem 3) Determine the maximum shear stress in a rectangular beam 6 in. wide and 8...

###### Question:

Problem 3) Determine the maximum shear stress in a rectangular beam 6 in. wide and 8 in. deep produced by a shear force of 38,400 lb.

#### Similar Solved Questions

##### A closed box is filled with dry ice at a temperature of -91.6 °C, while the...
A closed box is filled with dry ice at a temperature of -91.6 °C, while the outside temperature is 26.3°C. The box is cubical, measuring 0.364 m on a side, and the thickness of the walls is 4.14 x 10-2 m. In one day, 3.62 x 106 J of heat is conducted through the six walls. Find the thermal c...
##### 1 (50 pts). The following table is a set of data points. X у 2 15...
1 (50 pts). The following table is a set of data points. X у 2 15 5 12 9 6 12 7 (a) Use table and formulas to find the regression equation. (Note: Cannot use Excel or SPSS to do it.) (b) Estimate the value of y when x = 14....
##### 2nd question: A very long time after t = 0, when all currents and voltages in...
2nd question: A very long time after t = 0, when all currents and voltages in the circuit have settled down to steady values, the switch is opened again. What will be the current flowing through the 100-Ohm resistor 0.02 seconds after the switch is opened Make sure that you specific its direction (U...
##### For an infinite-source, single server system with an arrival rate of 15 customers per hour (Poisson)...
For an infinite-source, single server system with an arrival rate of 15 customers per hour (Poisson) and service time of 2 minutes per customer (exponential), the average number waiting in line to be served is: a. 0.1 b. 0.133 c. 0.50 d.0.250...
##### Classify the flange and web of I-section shown below in terms of compression resistance. It has...
Classify the flange and web of I-section shown below in terms of compression resistance. It has had stiffeners welded to the web and flanges. fy=355Mpa. 2. Classify the flange and the web of I-section shown below in terms of compression resistance. It has had stiffeners welded to the web and fla...
##### May absorb 11. Carbon monoxide rcacts with oxygen to form carbon dionide 2 CO(g)+ 0(g)2 COg)...
may absorb 11. Carbon monoxide rcacts with oxygen to form carbon dionide 2 CO(g)+ 0(g)2 COg) In a 1.00 L flask, 4.30 atm of CO reacts with 2.50 atm of O. Assuming that the temperature remains constant, what is the final pressure in the flask? a. 4.30 atm b. 6.80 atm 1.80 atm d. 4.65 atm 2.50 atm c. ...
##### Three charged particles are located at the corners of an equilateral triangle as shown in the...
Three charged particles are located at the corners of an equilateral triangle as shown in the figure below (let q 1.40 uC, and L HC charge. magnitude direction 0.910 m). Calculate the total electric force on the 7.00 。(counterclockwise from the +x axis) 7.00 μC 60.0 -4.00 μC...
##### Two parallel wires are each carrying 10 A and are separated by 4.0 m. If the...
Two parallel wires are each carrying 10 A and are separated by 4.0 m. If the currents are in opposite directions, what is the magnitude of the magnetic field halfway between them and which direction is it pointing? I already worked through the first part and got an answer of 2E-6 T, but I cannot fig...
##### What is the axis of symmetry and vertex for the graph y=2x^2+6x+4?
What is the axis of symmetry and vertex for the graph y=2x^2+6x+4?...


text:
Visualize Algorithms based on the backpropagation

Typical neural networks have mullions of parameters and it's quite difficult to visualize the process. In the article, we visualize training of the network that has only 2 parameters. It allows us to explore different training algorithms and see how it behaves during the training

These type of visualizations can provide us with useful insights about the training process.

Typical neural networks have mullions of parameters and it’s quite difficult to visualize its training process. In the article, we visualize training of the network that has only 2 parameters. It allows us to explore different training algorithms and see how it behaves during the training.

Checking data

First of all we need to define simple dataset which contains 6 points with two features.

import numpy as np

X = np.array([
[0.9, 0.3],
[0.5, 0.3],
[0.2, 0.1],
[0.7, 0.5],
[0.1, 0.8],
[0.1, 0.9],
])
y = np.array([
[1],
[1],
[1],
[0],
[0],
[0],
])

So we can make a scatter plot and look closer at this dots.

import matplotlib.pyplot as plt

plt.scatter(X[:, 0], X[:, 1], c=y, s=100)
plt.show()

From the figure above we can clearly see that all dots are linearly separable and we are able to solve this problem with simple perceptron. But the goal of this article is to make clear visualization of learning process for different algorithm based on the backpropagation method, so the problem has to be as simple as possible, because in other cases it will be complex to visualize.

So, since the problem is linear separable we can solve it without hidden layers in network. There are two features and two classes, so we can build network which will take 2 input values and will produce 1 output. We need just two weights, so we can visualize them in contour plot.

Initialize contour

I won’t add all code related to the plots building in the article. In case if you are interested you can check the main script here.

The plot above shows error rate that depends on the network’s weights. The best result corresponds to the smallest error value. The best weights combination for this problem should be near the bottom right corner in the white area.

Next, we are going to look at 5 algorithms based on the Backpropagation. They are:

• Momentum
• RPROP
• iRPROP+
• Gradient Descent + Golden Search

Let’s define start point for our algorithms. I’ve chosen the (-4, -4) point, because at this point network gives bad results and it will be interesting to observe the learning progress from a bad initialization point. In the script you can set up any other starting point you like.

This function will train the network until the error will be smaller than 0.125. Every network starts at place with coordinates (-4, -4) and finishes near the point with the error value lower than 0.125.

Visualize algorithms based on the Backpropagation

Gradient Descent got to the value close to 0.125 using 797 steps and this black curve is just tiny steps of gradient descent algorithm. We can zoom it and look even closer.

Now we can see some information about gradient descent algorithm. All steps for gradient descent algorithm have approximately similar magnitude. Their direction doesn’t vary because contours in the zoomed picture are parallel to each other and in it we can see that there are still a lot of steps that are needed to be made to achieve the minimum. Also we can see that small vectors are perpendicular to the contour.

The problem is that the step size is a very sensitive parameter for the gradient descent. In typical problem we won’t be able to visualize the learning progress and we won’t have an ability to see that our updates over the epochs are inefficient. For this result I’ve used step size equal to 0.3, but if we increased it to 10 we would reach our goal in 25 steps. I haven’t added any improvements to make a fair comparison to other algorithms in the summary chapter.

Momentum

Now let’s look at another very popular algorithm - Momentum.

Momentum got to the value close to 0.125 by 92 steps, which is more than 8 times less than for the gradient descent. The basic idea behind Momentum algorithm is that it accumulates gradients from the previous epochs. It means that if the gradient has the same direction after each epoch weight update vector magnitude will increase. But if the gradient stars changing its direction weight update vector magnitude will decrease. Check the figure again. Imagine that you’re standing at a skatepark. Than you throw a ball into a half-pipe in a way that makes it roll smoothly on the surface. While it rolls down the gravity force drags it down and it makes the ball roll faster and faster. Let’s get back to the Momentum algorithm and try to find these properties in the plot.

When we zoom the plot we can see that the direction for weight update vectors is almost the same and gradient’s direction doesn’t change after every epoch. In the picture above the vector which is the last on the right is bigger than the first one on the same plot on the left. Since it always moves forward it speeds up.

Let’s get back to the ball example. What happens when the ball reaches the pit of the half-pipe for the first time? Will it stop? Of course not. Ball gained enough speed for moving. So it will go up. But after that the ball will start to slow down and its amplitude will become smaller and smaller, because of the gravity force, that will continue to push it down to the pit and eventually it will stop to move. Let’s try to find the similar behavior in the same plot.

From the figure above it’s clear that weight update magnitude became smaller. Like a ball that slows down and changes its direction towards the minimum.

And finally to make it even more intuitive you can check weight update trajectory in 3D plot. It looks much more like the ball and half-pipe in skatepark analogy.

RPROP

Momentum makes fewer steps to reach the specified minimum point, but we still can do better. Next algorithm that we are going to check is RPROP.

This improvement looks impressive. Now we are able to see steps without zooming. We got almost the same value as before using just 20 steps, which is approximately 5 times less than Momentum and approximately 40 times less than Gradient Descent.

Now we are going to figure out what are the main features of RPROP. We can notice just by looking at the plot above RPROP has a unique step for each weight. There are just two steps for each weight in the input layer for this network. RPROP will increase the step size if gradient don’t change the sign compare to previous epoch, and it will decrease otherwise.

Let’s check a few first weight updates.

From the figure above you can see that first 11 updates have the same direction, so both steps increase their value after each iteration. For the first epoch steps are equal to the same value which we set up at network initialization step. In further iterations they increased by the same constant factor, so after six iteration they got bigger, but they are still equal because they move in one direction all the time.

Now let’s check the next epochs from the figure below. At the 12th epoch gradient changed the direction, but steps are still the same in value. But we can clearly see that gradient changed the sign for the second weight. RPROP updated the step after weight had updated, so the step for the second weight should be smaller for the 13th epoch.

Now let’s look at the 13th epoch. It shows us how gradient sign difference at the 12th epoch updated steps. Now the steps are not equal. From the picture above we can see that update on the second weight (y axis) is smaller than on the first weight (x axis).

At the 16th epoch gradient on y axis changed the sign again. Network decreased by constant factor and updated for the second weight at the 17th epoch would be smaller than at the 16th.

To train your intuition you can check the other epochs updates and try to figure out how steps depend on the direction.

iRPROP+

iRPROP+ is almost the same algorithm as RPROP except a small alteration.

As in RPROP algorithm iRPROP+ make exactly the same first 11 steps.

Now let’s look at the 12th step in the figure below.

Second weight (on the y axis) didn’t change the value. At the same epoch RPROP changed the gradient comparing to the previous epoch and just decreased step value after weight update whereas, iRPROP+ disabled weight update for current epoch (set it up to 0). And of course it also decreased the step for the second weight. Also you can find that vector for the 12th epoch that looks smaller than for the RPROP algorithm, because we ignored the second weight update. If we check the x axis update size we will find that it has the same value as in RPROP algorithm.

At 13th epoch network again included second weight into the update process, because compared to the previous epoch gradient didn’t change its sign.

The nice thing about this algorithm is that it tries to move in a new direction instead of going back and force and trying to redo updates from the previous epochs.

Summary

Summary table
Algorithm Number of epochs


text:
# Not so likely after all

Find the smallest positive integer $$N$$ with the following property:

Any subset of $$\{1, 2, 3, \ldots, 201\}$$ with $$N$$ elements must contain two numbers whose greatest common divisor is exactly $$2$$.

×


text:
Delta function charge densities

1. Feb 2, 2010

kreil

1. The problem statement, all variables and given/known data
Using Dirac delta function in the appropriate coordinates, express the following charge distributions as three-dimensional charge densities p(x).

(a) In spherical coordinates, a charge Q uniformly distributed over a spherical shell of radius a.

(b) In cylindrical coordinates, a charge $\lambda$ per unit length uniformly distributed over a cylindrical surface of radius b.

(c) In cylindrical coordinates, a charge Q spread uniformly over a flat circular disc of negligible thickness and radius R.

(d) The same as in (c), but using spherical coordinates.

Note that
$$\delta(x-x')= \frac{1}{|J(x_i, \zeta_i)|} \delta( \zeta_1 - \zeta_1')\delta( \zeta_2 - \zeta_2')\delta( \zeta_3 - \zeta_3')$$

where $J(x_i, \zeta_i)$ is the Jacobian relating cartesian coordinates $(x_1,x_2,x_3)$ to new coordinates $(\zeta_1,\zeta_2,\zeta_3)$.

2. Relevant equations

$$\int_{- \infty}^{+ \infty} f(x) \delta(x-a) dx = f(a)$$

3. The attempt at a solution

For some reason I can't wrap my head around how to methodically do this problem. Let's start by discussing part (a). Here is what I have:

(a)

The definition of charge density is: $$\rho = \frac{TotCharge}{TotArea}$$

In this case then, we get: $$\rho = \frac{Q}{4 \pi a^2}$$.

However, this charge is localized to the surface of the sphere where r=a, so:

$$\rho(r)=\frac{Q}{4 \pi a^2} \delta(r-a)$$

is this a correct answer? it doesnt seem to be three dimensional, but then again $\theta$ and $\phi$ seem like irrelevant values since the charge is localized to the surface where r=a. Doing a check integration of the charge density in 3d space produces the total charge Q, so I have reason to believe it is correct..

As far as the rest of them go, I could use some help as to how to get started/use that jacobian equation.

2. Feb 2, 2010

Oh, really?

3. Feb 2, 2010

kreil

lol just trying to be clear.

4. Feb 2, 2010

gabbagabbahey

The point of my question was to intimate that the definition you gave is incorrect, and is leading to your confusions.....I suggest you look up the definition of "volume charge density".

5. Feb 2, 2010

kreil

im aware of that, but in part a it is a surface charge density isn't it?

6. Feb 2, 2010

gabbagabbahey

No, in all 4 parts of the question your are asked to find the volume charge density....in part (a), the surface charge density is finite everywhere, so no delta function would be needed....however, the volume charge density is not finite everywhere, and yet the total charge is finite and so a delta function is needed.
So, what is the general definition of volume charge density?

7. Feb 2, 2010

kreil

charge/volume, so for a it would be $$\rho = \frac{3Q}{4 \pi a^3}$$

8. Feb 2, 2010

gabbagabbahey

Not quite, and no.

The definition of volume charge density, in a region $\mathcal{V}$, is given mathematically by the integral

$$Q=\int_{\mathcal{V}}\rho(\textbf{x})d^3\textbf{x}$$

(using the notation of Jackson)

It is only when the charge density is uniform over the entire region, that this definition is equivalent to $\rho=\text{charge}/\text{volume}$.

Is the charge density of a spherical shell really uniform over the entire volume of the sphere?

9. Feb 2, 2010

kreil

ohh ok, i didnt think to use that definition.

I suppose the charge density would not be uniform since the charge is localized to the surface and is not present inside/outside the sphere.

how do i proceed using that definition?

10. Feb 2, 2010

gabbagabbahey

Okay, so you know the following things:

(1)The charge density is zero everywhere except on the surface ($r=a$) where it is uniformly distributed
(2)The total charge is Q

Now, what else can we say?

Let's look at te above definition:

$$Q=\int_{\mathcal{V}}\rho(\textbf{x})d^3\textbf{x}\implies \rho(\textbf{x})=\frac{dQ}{d^3\textbf{x}}$$

What is the volume element $d^3\textbf{x}$ in spherical coordinates?

11. Feb 2, 2010

kreil

$$d^3 \textbf{x}=r^2 sin \theta dr d\theta d\phi$$

Last edited: Feb 2, 2010
12. Feb 2, 2010

gabbagabbahey

Right, so

$$\rho(\textbf{x})=\frac{dQ}{r^2\sin\theta dr d\theta d\phi}$$

Now, clearly dQ will be zero everywhere except on the surface, and hence so will $\rho$.....so let's look at the surface...It has finite extent in both the azimuthal and polar dir ections ($\phi$ varies from zero to $2\pi$ and $\theta$ varies from zero to $\pi$ over the surface)....But what about the radial direction...for an infinitesimally thin surface, doesn't it have zero extent (meaning that $dr=0$)?....What does that tell you about $\rho$ at the surface?

13. Feb 2, 2010

kreil

so rho is zero everywhere except at the surface (r=a), where it blows up to infinity (since if dr=0 p=dQ/0) so it can be identified with a delta function of the form $\delta(r-a)$?

14. Feb 2, 2010

gabbagabbahey

You've listed two of the three qualities of that delta function (zero everywhere except at $r=a$, where it is infinite)....what about the 3rd property of a delta function....does the charge density satisfy that property as well?

If so, then surely you can say that it must be of the form $\rho(\textbf{r})=f(\textbf{r})\delta(r-a)$ where $f$ is some well behaved function....right?

15. Feb 2, 2010

kreil

$$\int_{-\infty}^{+\infty} \delta(r-a)dr = 1$$

and we know $$\frac{1}{Q} \int_{\mathcal{V}}\rho(\textbf{x})d^3\textbf{x}=1$$

so it satisfies the property.

Therefore, $$\rho(\textbf{r})=f(\textbf{r})\delta(r-a)$$.

Now, how do we find f(r)? Would we plug this into the above integral and solve for f(a)?

16. Feb 2, 2010

gabbagabbahey

The more general property I was looking for from you, is

$$\int_{-\infty}^{+\infty} f(r)\delta(r-a)dr = f(a)$$

which means that the integral of the charge density over any volume enclosing the surface must be finite (despite the density itself being infinite)

Well, plug this form for $\rho[/tex] into this integral you have and what do you get?...What is the simplest [itex]f(\textbf{r})$ that satisfies this relation?

17. Feb 2, 2010

kreil

$$\int_{all}f(r)r^2\delta(r-a)dr \int_0^{\pi}sin\theta d \theta \int_0^{2 \pi}d \phi=Q$$

$$Q= 4 \pi a^2 f(a) \implies f(a)=\frac{Q}{4 \pi a^2}$$

So $$\rho(r)= \frac{Q}{4 \pi r^2}\delta(r-a)$$....?

18. Feb 2, 2010

gabbagabbahey

Yup, $\rho(r)= \frac{Q}{4 \pi a^2}\delta(r-a)[/tex] would also work...in fact, there are several [itex]f(\textbf{r})$ that would work, but all are mathematically equivalent.

Now, apply similar reasoning to the rest f the charge distributions....se what you can come up with.

19. Feb 2, 2010

kreil

thanks a lot for all your help and patience

20. Aug 25, 2011

jfy4

Hi,

Using what was talked about in this thread I have been able to do (a) and (b), but I am still stuck on (c) and (d). Its the fact that there is continuous charge distribution from the center of the disc out to a finite radius. I'm not sure how to deal with that part. From what I gather, I need a delta function for the fact that the disc is flat in the z-direction, $\delta(z)$, but also there is an infinity at $r=0$. but I'm not sure how to add a delta function such that I still get a continuous range of values out to a finite radius.

Any help would be great, thanks,


text:
Math Help - Another integral, general power formula.

1. Another integral, general power formula.

Hi thanks for the help lately its been good. I don't post these unless a day goes by without a solution.

28-1-20

Integrate:

(int) 2 sqrt(1- e^(-x)) (e^-x) dx

therefore,
u = 1 - e^(-x)
du = - e^(-x)

General power form:
[1 - e^(-x)]^(3/2) / (3/2) + C
=
4/3 [1 - e^(-x)]^(3/2) + C

however here is the ti-89 answer:

somehow something has been added in the nominator.
how did I go wrong? Thanks!

2. Re: Another integral, general power formula.

Hello, togo!

$\int \sqrt{1-e^{-x}}\cdot e^{-x}\,dx$

Therefore:

$u \:=\: 1 - e^{-x}$

$du \:=\: - e^{-x}dx$ . . . . no

$\text{If }u \:=\:1-e^{-x}$

. . $\text{then: }\:du \:=\:\left[0-e^{-x}(\text{-}1)\right]dx \:=\:e^{-x}dx$

3. Re: Another integral, general power formula.

but, that doesn't really change what I did at all

its the added e^(-3x/2) that is confusing me in the nominator

4. Re: Another integral, general power formula.

Originally Posted by togo
but, that doesn't really change what I did at all

its the added e^(-3x/2) that is confusing me in the nominator
Well it DOES change what you did, because if you followed your own logic, you should have ended up with a negative answer. I'm not sure where your negative has gone.

5. Re: Another integral, general power formula.

I have been having trouble with losing the negative and have retraced it twice in homework the last couple days.

This is something I can do myself and I'm not sure what to do in order to complete the answer besides this. I'll rework the question soon and hopefully it does give me the answer shown on the ti-89.

6. Re: Another integral, general power formula.

ya that change does not give me a complete answer.


text:
kidzsearch.com > wiki   Explore:web images videos games

# Chevrolet Volt

Chevrolet Volt

Chevrolet Volt is an American plug-in hybrid car. The car was first produced in 2011[1][2] and is manufactured by Chevrolet division of General Motors. This vehicle travels 25 to 50 miles (38 miles average) on electric power alone. After the battery is depleted, it switches to the extended range mode.[1] In the United States, the car has a suggested price of $34,185, but qualifies for a$7500 tax credit. It can carry up to four people (including the driver).

## Design

The Volt has 16.5 KWh of lithium batteries and a 1.4 liter engine using premium fuel.[3] The batteries have enough power to drive the car normally (it can go from 0 to 60 mph in 9 seconds). While the batteries have power, the car is like an electric car.

When the batteries run out of charge, they can be recharged from a special electrical outlet, or the car can turn on the gas engine. The gas engine and electric motors combine in a special type of transmission called a planetary gear,[1] like the Toyota Prius. This allows the gas engine to drive the car or to change its power to electricity. The electricity can charge the battery or power the electric motor.

## Fuel efficiency

How much gas is used depends on how the car is driven and when the batteries are recharged. It is possible to only use electric power and never use gasoline. The EPA estimates the average person will get 98 miles per gallon (including the equivalent cost of electricity).[3]

## Recharging

The batteries can be recharged in 10 to 16 hours using a 120V charging unit (most circuits in a house are at 120V). A 240V charging unit will recharge the car batteries in four hours.[3] The batteries can also be charged from the gas engine or from the regenerative brakes, which turn the energy of the moving car into electricity.

## References

1. Matthe, Roland; Eberle, Ulrich (2014-01-01). "The Voltec System - Energy Storage and Electric Propulsion". Retrieved 2014-05-04.
2. "Chevy Volt review". plug-in cars.com. Retrieved 5 October 2013.
3. "Volt electric car". Chevrolet. Retrieved 5 October 2013.


text:
# Align glosses in more than one language with gb4e

I'm using the package gb4e to gloss my linguistic examples. It nicely aligns the words in the original language with its glosses. But this alignment does not extend beyond the first line of glosses. If I want to gloss the words in more than one language, there will be no alignment beyond the first line of glosses. How can I enforce such alignment?

\documentclass{article}
\usepackage{gb4e}
\begin{document}
\begin{exe}
\ex
\gll gef hundinum matinn\\
gje hunden maten\\
give {the dog} {the food}\\
gib {dem Hunde} {das Essen}\\
\end{exe}
\end{document}

• Manually should be no problem, but I guess you want it automated (with cookies on the side). – Werner Jan 26 '14 at 19:30
• @Werner Manual solutions will also be appreciated! :) – Sverre Jan 26 '14 at 19:35

The cgloss4e package, which is loaded by gb4e is able to account for an example with two lines of glosses automatically, using the \glll (note the extra l) macro:

\documentclass{article}
\usepackage{gb4e}
\begin{document}
\begin{exe}
\ex
\glll gef hundinum matinn\\
give {the dog} {the food}\\
gib {dem Hunde} {das Essen}\\
\end{exe}
\end{document}

However, for your example, you require yet one more line of gloss, for which the package is not actually designed. But it's not to hard to add the relevant code. Here I've created a \gliv command to introduce an example with 3 extra gloss lines.

\documentclass{article}
\usepackage{gb4e}
\makeatletter
\newbox\linefour
\newbox\wordfour
\def\gliv%                 % Introduces 4-line text-and-gloss.
{\begin{flushleft}
\ifx\@gsingle1%
\vskip\baselineskip\def\baselinestretch{1}%
\@selfnt\vskip-\baselineskip\fi%
\bgroup
\foursent
}
\@ifundefined{eachwordfour}{\let\eachwordfour=\rmfamily}{\relax}
\gdef\foursent#1\\ #2\\ #3\\ #4\\{% #1 = first line, #2 = second line, #3 = third, #4 = fourth
\getwords(\lineone,\eachwordone)#1 \\%
\getwords(\linetwo,\eachwordtwo)#2 \\%
\getwords(\linethree,\eachwordthree)#3 \\%
\getwords(\linefour,\eachwordfour)#4 \\%
\loop\lastword{\eachwordone}{\lineone}{\wordone}%
\lastword{\eachwordtwo}{\linetwo}{\wordtwo}%
\lastword{\eachwordthree}{\linethree}{\wordthree}%
\lastword{\eachwordfour}{\linefour}{\wordfour}%
\global\setbox\gline=\hbox{\unhbox\gline
\hskip\glossglue
\vtop{\box\wordone   % vtop was vbox
\nointerlineskip
\box\wordtwo
\nointerlineskip
\box\wordthree
\nointerlineskip
\box\wordfour
}%
}%
\testdone
\ifnotdone
\repeat
\egroup % matches \bgroup in \gloss
\gl@stop}
\makeatother

\begin{document}
\begin{exe}
\ex
\gliv gef hundinum matinn\\
gje hunden maten\\
give {the dog} {the food}\\
gib {dem Hunde} {das Essen}\\
\end{exe}
\end{document}

A manual way of achieving this is possible via tabbing:

\documentclass{article}
\usepackage{gb4e}
\begin{document}
\begin{exe}
\ex \begin{tabbing}
give \= dem Hunde \= das Essen \kill
gef \> hundinum \> matinn \\
gje \> hunden \> maten\\
give \> the dog \> the food \\
gib \> dem Hunde \> das Essen
\end{tabbing}
\end{exe}
\end{document}

The \kill line contains the longest word across each language translation to mark the tabbing points with \=.

• This solution is really only practical for very short lines. One of the things that the glossing macros do is automatically line up each word independent of the length of the example, so that the glossed lines will wrap as needed. – Alan Munn Jan 27 '14 at 2:51
• @AlanMunn: True. You've done a great job at extending this. – Werner Jan 27 '14 at 5:16