text:
الرئيسية An Introduction to Wavelet Analysis

# An Introduction to Wavelet Analysis

"D. Walnut's lovely book aims at the upper undergraduate level, and so it includes relatively more preliminary material...than is typically the case in a graduate text. It goes from Haar systems to multiresolutions, and then the discrete wavelet transform... The applications to image compression are wonderful, and the best I have seen in books at this level. I also found the analysis of the best choice of basis and wavelet packet especially attractive. The later chapters include MATLAB codes. Highly recommended!" --- Bulletin of the AMS
"[This text] is carefully prepared, well-organized, and covers a large part of the central theory...[there are] chapters on biorthogonal wavelets and wavelet packets, topics which are rare in wavelet books. Both are important, and this feature is an extra argument in favour of [this] book...the material is accessible [even] to less advanced readers...the book is a nice addition to the series." --Zentralblatt Math
"This book can be recommended to everyone, especially to students looking for a detailed introduction to the subject." --Mathematical Reviews
"This textbook is an introduction to the mathematical theory of wavelet analysis at the level of advanced calculus. Some applications are described, but the main purpose of the book is to develop---using only tools from a first course in advanced calculus---a solid foundation in wavelet theory. It succeeds admirably.... Part I of the book contains 112 pages of preliminary material, consisting of four chapters on 'Functions and Convergence,' 'Fourier Series,' 'Fourier Transforms,' and 'Signals and Systems....' This preliminary material is so well written that it could serve as an excellent supplement to a first course in advanced calculus.... The heart of the book is Part III: 'Orthonormal Wavelet bases.' This material has become the canonical portion of wavelet theory. Walnut does a first-rate job explaining the ideas here.... Ample references are supplied to aid the reader.... There are exercises at the end of each section, 170 in all, and they seem to be consistent with the level of the text....To cover the whole book would require a year. An excellent one-semester course could be based on a selection of chapters from Parts II, III, and V." --SIAM Review
"An Introduction to Wavelet Analysis" provides a comprehensive presentation of the conceptual basis of wavelet analysis, including the construction and application of wavelet bases.
The book develops the basic theory of wavelet bases and transforms without assuming any knowledge of Lebesgue integration or the theory of abstract Hilbert spaces. The book elucidates the central ideas of wavelet theory by offering a detailed exposition of the Haar series, and then shows how a more abstract approach allows one to generalize and improve upon the Haar series. Once these ideas have been established and explored, variations and extensions of Haar construction are presented. The mathematical prerequisites for the book are a course in advanced calculus, familiarity with the language of formal mathematical proofs, and basic linear algebra concepts.
Features:
* Rigorous proofs have consistent assumptions about the mathematical background of the reader (does not assume familiarity with Hilbert spaces or Lebesgue measure).
* Complete background material is offered on Fourier analysis topics.
* Wavelets are presented first on the continuous domain and later restricted to the discrete domain for improved motivation and understanding of discrete wavelet transforms and applications.
* Special appendix, "Excursions in Wavelet Theory," provides a guide to current literature on the topic
* Over 170 exercises guide the reader through the text.
"An Introduction to Wavelet Analysis" is an ideal text/reference for a broad audience of advanced students and researchers in applied mathematics, electrical engineering, computational science, and physical sciences. It is also suitable as a self-study reference for professionals.
السنة: 2001
الطبعة: 1
الناشر: Birkhäuser Boston
اللغة: english
الصفحات: 452
ISBN 10: 0817639624
ISBN 13: 9780817639624
File: DJVU, 6.74 MB

You can write a book review and share your experiences. Other readers will always be interested in your opinion of the books you've read. Whether you've loved the book or not, if you give your honest and detailed thoughts then people will find new books that are right for them.
1

### Differential Equations With Symbolic Computation

السنة: 2005
اللغة: english
File: PDF, 9.94 MB
2

### Consciousness. A Mathematical Treatment of the Global Neuronal Workspace Model

السنة: 2005
اللغة: english
File: PDF, 4.77 MB
..11 ::.1"

... Il "- . . ."';." -- ..:

.,.

.-..

.. ...

-

= -

-'- --

--- ,- --- ::-

.-

AN

.... ...'Ir' T R 0 -.. I. T I 0 -.. ,.. -. J ....-. r. Oil'

a Analysis

.

David F. Walnut

Birkhiiuser

Applied and Numerical Harmonic Analysis

Series Editor John J. Benedetto University of Maryland

Akram Aldroubi NIH, Biomedical Engineering/ Instrumentation

Ingrid Daubechies Princeton University

Christopher Heil Georgia Institute of Technology

James McClellan Georgia Insitute of Technology

Michael Unser NIH, Biomedical Engineering/ Instrumentation

M. Victor Wickerhauser Washington University

Douglas Cochran Arizona State University

Hans G. Feichtinger University of Vienna

Murat Kunt Swiss Federal Institute of Technology, Lausanne

Wim Sweldens Lucent Technologies Bell Laboratories

Martin Vetterli Swiss Federal Institute of Technology, Lausanne

i

Applied and Numerical Harmonic Analysis

J.M. Cooper: Introduction to Partial Differential Equations with MA TLAB (ISBN 0-8176-3967-5) C.E. D'Attellis and E.M. Fermindez-Berdaguer: Wavelet Theory and Har- monic Analysis in Applied Sciences (ISBN 0-8176-3953-5) H.G. Feichtinger and T Strohmer: Gabor Analysis and Algorithms (ISBN 0-8176-3959-4 ) TM. Peters, J.H.T Bates, G.B. Pike, P. Munger, and J.C. Williams: Fourier Transforms and Biomedical Engineering (ISBN 0-8176-3941-1) A.I. Saichev and W.A. Woyczynski: Distributions in the Physical and Engi- neering Sciences (ISBN 0-8176-3924-1) R. Tolimierei and M. An: Time-Frequency Representations (ISBN 0-8176- 3918-7) G.T Herman: Geometry of Digital Spaces (ISBN 0-8176-3897-0) A. Prochazka, J. Uhlir, P.J.W. Rayner, and N.G. Kingsbury: Signal Analy- sis and Prediction (ISBN 0-8176-4042-8) J. Ramanathan: Methods of Applied Fourier Analysis (ISBN 0-8176-3963-2) A. Teolis: Computational Signal Processing with Wavelets (ISBN 0-8176- 3909-8) W.O. Bray and G.V. Stanojevi6: Analysis of Divergence (ISBN 0-8176- 4058-4 ) G.T Herman and A. Kuba: Discrete Tomography (ISBN 0-8176-4101-7) J.J. Benedetto and P.J.S.G. Ferreira (Eds.): Modern Sampling Theory (ISBN 0-8176-4023-1) P. Das, A. Abbate, and C. DeCusatis: Wavelets and Subbands (ISBN 0- 8176-4136-X) L. Debnath: Wavelet Transforms and Time-Frequency Signal Analysis (ISBN 0-8176-4104-1) K. Grochenig: Foundations of Time-Frequency Analysis (ISBN 0-8176- 4022-3) D.F. Walnut: An Introduction to Wavelet Analysis (ISBN 0-8176-3962-4)

David F. Walnut

An Introdution to Wavelet Analysis

With 88 Figures

B ir kha user Boston. Basel . Berlin

David F. Walnut Department of Mathematical Sciences George Mason University Fairfax, VA 22030 USA

Library of Congress Cataloging-in-Publication Data Walnut, David F. An introduction to wavelet. analysis / David F. Walnut p. ern. (Applied and numerical harmonic analysis) Includes bibliographical references and index. ISBN 0-B176-3962-4 (alk. paper) 1. Wavelets (Mathematics) 1. Title. II. Series. QA403.3 .W335 2001 515'.2133 dc21

2001025:367 eIP

Printed on acid-free paper. @ 2002 Birkhiiuser Boston

Q?) ,R) Birkhauser l!{VJ

All right.s reserved. This work may not be translated or copied in whole or in part without. the written lwnnission of the publisher (Birkhiiuser Boston, c/o Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection wit.h reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer soft.ware, or by similar or dissimilar methodol- ogy now known or hereafter developed is forbidden. The use of general descript.ive names, trade names, tradcmarks, etc., in this publication, even if the fonner are not especially ident.ified, is not t.o be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.

ISBN 0-B176-3962-1 ISBN :3-7643-3962-4

SPIN 10574019

Product.ion managed by Louise Farkas; manufacturing supervised by Jacqui Ashri. Typeset by the author in LaTeX2e. Printed and bound by Edwards Brothers, Inc., Ann Arbor, M1. Printed in the United States of America.

9 B 7 6 5 1 321

Birkhiiuser Boston Basel Berlin J1 meTn{W'r oj l3edelsTnannSpT'ingeT Science+B'/1,s,;npss JIJed-ia GTnbH

To my parents

and to Megan

Unless the LORD builds the house, its builders labor in vain.

- Psalm 121: 1 a (NIV)

Contents

Preface xiii I Preliminaries 1 1 Functions and Convergence 3 1.1 Functions 3 1.1.1 Bounded (L =) Functions 3 1.1.2 Integrable (L 1 ) Functions 3 1.1.3 Square Integrable (L 2 ) Functions 6 1.1.4 Differentiable (en) Functions 9 1.2 Convergence of Sequences of Functions . 11 1.2.1 Numerical Convergence 11 1.2.2 Pointwise Convergence . 13 1.2.3 Uniform (L =) Convergence 14 1.2.4 Mean (L 1) Convergence 17 1.2.5 Mean-square (L2) Convergence 19 1.2.6 Interchange of Limits and Integrals . 21 2 Fourier Series 27 2.1 Trigonometric Series 27 2.1.1 Periodic Functions 27 2.1.2 The Trigonometric System 28 2.1.3 The Fourier Coefficients 30 2.1.4 Convergence of Fourier Series 32 2.2 Approximate Identities . 37 2.2.1 :Motivation from Fourier Series 38 2.2.2 Definition and Examples . 40 2.2.3 Convergence Theorems. 42 2.3 Generalized Fourier Series 47 2.3.1 Orthogonality. 47 2.3.2 Generalized Fourier Series 49 2.3.3 Completeness 52 3 The Fourier Transform 59 3.1 Nlotivation and Definition 59 3.2 Basic Properties of the Fourier Transform 63 3.3 Fourier Inversion 65

viii Contents

3.4 Convolution................ 3.5 Plancherel's Formula . . . . . . . . . . . 3.6 The Fourier Transform for L 2 Functions 3.7 Smoothness versus Decay . . . . . . . . 3.8 Dilation, Translation, and "NIodulation . 3.9 Bandlimitecl Functions and the Sampling Formula

68 72 75 76 79 81

4 Signals and Systems 4.1 Signals.............. 4.2 Systems . . . . . . . . . . . . . 4.2.1 Causality and Stability 4.3 Periodic Signals and the Discrete Fourier Transform 4.3.1 The Discrete Fourier Transform. 4.4 The Fast Fourier Transform 4.5 L 2 Fourier Series . . . . . . . . . . . . .

87 88 90 95 101 102 107 109

II The Haar System

113

5 The Haar System 115 5.1 Dyadic Step Functions . . . . . . . . . . . 115 5.1.1 The Dyadic Intervals. . . . . . . . 115 5.1.2 The Scale j Dyadic Step Functions 116 5.2 The Baar System. . . . . . . . . . . . . . 117 5.2.1 The Baal' Scaling Functions and the Baar Functions . . . . . . . . . . . 11 7 5.2.2 Orthogonality of the Baar System 118 5.2.3 The Splitting Lemma ....... 120 5.3 Baar Bases on [0,1]. . . . . . . . . . . . . 122 5.4 Comparison of Baar Series with Fourier Series. 127 5.4.1 Representation of Functions with Small Support 128 5.4.2 Behavior of Baar Coefficients Near Jump Discontinuities. . . . . . . . . . . . 130 5.4.3 Baar Coefficients and Global Smoothness 132 5.5 Baar Bases on R . . . . . . . . . . . . . . . . . . 133 5.5.1 The Approximation and Detail Operators 134 5.5.2 The Scale J Baar System on R 138 5.5.3 The Baar System on R 138

6 The Discrete Haar Transform 141 6.1 Motivation ................... 141 6.1.1 The Discrete Baar Transform (DBT) . 142 6.2 The DBT in Two Dimensions . . . . . . . . . 146 6.2.1 The Row-wise and Column-wise Approximations and Details . . . . . . . . . . . . . . . . . . . . . 146

Contents IX

6.2.2 The DHT for lVIatrices . . . . 6.3 Image Analysis with the DHT. . . . 6.3.1 Approximation and Blurring 6.3.2 Horizontal, Vertical, and Diagonal Edges 6.3.3 "Naive" Image Compression. . . . . . . .

147 150 151 153 154

III Orthonormal Wavelet Bases

161

7 Multiresolution Analysis 163 7.1 Orthonormal Systems of Translates. 164 7.2 Definition of Multiresolution Analysis . 169 7.2.1 Some Basic Properties of MRAs 170 7.3 Examples of Multiresolution Analysis. 174 7.3.1 The Haar MRA. . . . . . . 174 7.3.2 The Piecewise Linear l\1RA 174 7.3.3 The Bandlimited l\1RA .. 179 7.3.4 The Meyer MRA . . . . . . 180 7.4 Construction and Examples of Orthonormal Wavelet Bases. . . . . . . . . . . . . 185 7.4.1 Examples of Wavelet Bases . 186 7.4.2 Wavelets in Two Dimensions 190 7.4.3 Localization of Wavelet Bases 193 7.5 Proof of Theorem 7.35 . . . . . . . . 196 7.5.1 Sufficient Conditions for a Wavelet Basis. 197 7.5.2 Proof of Theorem 7.35 . . . . . . . . . 199 7.6 Necessary Properties of the Scaling Function 203 7.7 General Spline Wavelets . . . . . . . . . . . 206 7.7.1 Basic Properties of Spline Functions 206 7.7.2 Spline Multiresolution Analyses. 208

8 The Discrete Wavelet Thansform 215 8.1 Motivation: From MRA to a Discrete Transform 215 8.2 The Quadrature lVIirror Filter Conditions . . . . 218 8.2.1 Motivation from MRA . . . . . . . . . . . 218 8.2.2 The Approximation and Detail Operators and Their Adjoints . . . . . . . . . . . . . . . . . . 221 8.2.3 The Quadrature Mirror Filter (QMF) Conditions. 223 8.3 The Discrete Wavelet Transform (DWT) 231 8.3.1 The DWT for Signals . . . . . . . . . . . . . 231 8.3.2 The DWT for Finite Signals. . . . . . . . . . 231 8.3.3 The DWT as an Orthogonal Transformation 232 8.4 Scaling Functions from Scaling Sequences 236 8.4.1 The Infinite Product Formula 237 8.4.2 The Cascade Algorithm . . . . . . 243

x

Contents

8.4.3

The Support of the Scaling Function

245

9 Smooth, Compactly Supported Wavelets 249 9.1 Vanishing l\loments. . . . . . . . . . . . . 249 9.1.1 Vanishing Moments and Smoothness . . 250 9.1.2 Vanishing l\1oments and Approximation 254 9.1.

3 Vanishing Moments and the Reproduction of Polynomials . . . . . . . . . . . . . . . . 257 9.1.4 Equivalent Conditions for Vanishing l\loments . 260 9.2 The Daubechies Wavelets . . . . . . 264 9.2.1 The Daubechies Polynomials . 264 9.2.2 Spectral Factorization . . . . . 269 9.3 Image Analysis with Smooth Wavelets 277 9.3.1 Approximation and Blurring . 278 9.3.2 "Naive" Image Compression with Smooth Wavelets . . . . . . . . . . . . . . . . . . . . 278

IV Other Wavelet Constructions

287

10 Biorthogonal Wavelets 10.1 Linear Independence and Biorthogonality 10.2 Riesz Bases and the Frame Condition .. 10.3 Riesz Bases of Translates ......... 10.4 Generalized Multiresolution Analysis (GMRA) 10.4.1 Basic Properties of GMRA . . . . . . . 10.4.2 Dual GMRA and Riesz Bases of Wavelets 10.5 Riesz Bases Orthogonal Across Scales ...... 10.5.1 Example: The Piecewise Linear GMRA . 10.6 A Discrete Transform for Biorthogonal Wavelets 10.6.1 Motivation from G MRA . . . . . . . . 10.6.2 The QMF Conditions .. . . . . . . . 10.7 Compactly Supported Biorthogonal Wavelets 10.7.1 Compactly Supported Spline Wavelets 10.7.2 Symmetric Biorthogonal Wavelets 10.7.3 Using Symmetry in the DWT

289 289 290 293 ;300 301 302 311 313 315 315 317 319 320 324 328

11 Wavelet Packets 11.1 Motivation: Completing the Wavelet Tree 11.2 Localization of Wavelet Packets . 11.2.1 Time/Spatial Localization . . . . . 11.2.2 Frequency Localization ...... 11.3 Orthogonality and Completeness Properties of Wavelet Packets ................. 11.3.1 Wavelet Packet Bases with a Fixed Scale

335 335 337 337 338

346 347

11.3.2 Wavelet Packets with "Mixed Scales. . . . . 11.4 The Discrete Wavelet Packet Transform (DWPT) . 11.4.1 The DWPT for Signals 11.4.2 The DWPT for Finite Signals. . . . . 11.5 The Best-Basis Algorithm . . . . . . . . . . . 11.5.1 The Discrete Wavelet Packet Library. 11.5.2 The Idea of the Best Basis. . 11.5.3 Description of the Algorithm . . . . .

V Applications

12 Image Compression 12.1 The Transform Step . . . . . . . . . 12.1.1 Wavelets or Wavelet Packets? 12.1.2 Choosing a Filter. 12.2 The Quantization Step. . 12.3 The Coding Step . . . . . 12.3.1 Sources and Codes 12.3.2 Entropy and Information 12.3.3 Coding and Compression 12.4 The Binary Huffman Code. . . . 12.5 A :Model Wavelet Transform Image Coder 12.5.1 Examples . . . . . . . . . . . . . .

13 Integral Operators 13.1 Examples of Integral Operators . . . . . . . . . . . 13.1.1 Sturm-Liouville Boundary Value Problems. 13.1.2 The Hilbert Transform. 13.1.3 The Radon Transform . . . . . . 13.2 The BCR Algorithm . . . . . . . . . . . 13.2.1 The Scale j Approximation to T 13.2.2 Description of the Algorithm . .

VI Appendixes

Contents

Xl

350 354 354 354 357 357 360 363

369

371 372 372 373 373 375 376 378 380 385 387 388

397 397 397 402 406 414 415 418

423

A Review of Advanced Calculus and Linear Algebra 425 A.1 Glossary of Basic Terms from Advanced Calculus and Linear Algebra . . . . . . . . . . . . . . . 425 A.2 Basic Theorems from Advanced Calculus 431

B Excursions in Wavelet Theory 433 B.1 Other Wavelet Constructions 433 B.1.1 M-band Wavelets 433

XlI Contents

B.1.2 Wavelets with Rational Noninteger Dilation Factors .......... 434 B.1.3 Local Cosine Bases. . . . . . . . . 434 B.1.4 The Continuous Wavelet Transform 435 B.1.5 Non-J'vIRA Wavelets 436 B.1.6 Multiwavelets.. . . . 436 B. 2 Wavelets in Other Domains . 437 B.2.1 Wavelets on Intervals 437 B.2.2 Wavelets in Higher Dimensions 438 B.2.3 The Lifting Scheme 438 B.3 Applications of Wavelets. . . . . . 439 B.3.1 Wavelet Denoising . . . . . 439 B.3.2 Multiscale Edge Detection. 439 B.3.3 The FBI Fingerprint Compression Standard. 439

C References Cited in the Text

441

Index

445

Preface

xiv Preface

tion of wavelets and wavelet bases in the continuous domain and use this to motivate the discrete theory, even though the discrete theory hangs to- gether in its own right and is easy to understand. This can be frustrating for the student whose primary interest is in applications, but I believe that a better understanding of applications can ultimately be achieved by doing things in this order. (3) Prepare readers to explore wavelet theory on their own. Wave- lets is too broad a subject to cover in a single book and is most interesting to study when the students have a particular interest in what they are study- ing. In choosing what to include in the book, I have tried to ensure that students are equipped to pursue more advanced topics on their own. I have included an appendix called Excursions in Wavelet Theory (Appendix B) that gives some guidance toward what I consider to be the most readable articles on some selected topics. The suggested topics in this appendix can also be used as the basis of semester projects for the students.

Structure of the Book

The book is divided into five parts: Preliminaries, The Ham' System, Mul- tiresolution Analysis and Orthonormal Wavelet Bases, Other Wavelet Con- structions, and Applications.

Preliminaries

Wavelet theory is really very hard to appreciate outside the context of the language and ideas of Fourier Analysis. Chapters 1-4 of the book provide a background in some of these ideas and include everything that is sub- sequently used in the text. These chapters are designed to be more than just a reference but less than a "book-within-a-book" on Fourier analysis. Depending on the background of the reader or of the class in which this book is being used, these chapters are intended to be dipped into either superficially or in detail as appropriate. Naturally there are a great many books on Fourier analysis that cover the same material better and more thoroughly than do Chapters 1-4 and at the same level (more or less) of mathematical sophistication. I will list some of my favorites below. Walker, Fourier Analysis; Kammlcr, A First Course in Fouder AnalYfiis: Churchill and Brown, Fourier Sericfi and Boundary Value Problems: Dym and McKean, Fourier Series and Integrals; Korner, Fourier Analysis; and Benedetto, Harmonic Analysis and Applications.

The Haar System Chapters 5 and 6 provide a self-contained exposition of the Haar system. the earliest example of an orthonormal wavelet basis. These chapters could

Preface xv

be presented as is in a course on advanced calculus, or an undergraduate Fourier analysis course. In the context of the rest of the book, these chap- ters are designed to motivate the search for more general wavelet bases with different properties, and also to illustrate some of the more advanced concepts such as multiresolution analysis that are used throughout the rest of the book. Chapter 5 contains a description of the Baal' basis on [0, 1] and on R, and Chapter 6 shows how to implement a discrete ver:::;ion of the Haar basis in one and two dimensions. Some examples of images analyzed with the Haar wavelet are also included.

Orthonormal Wavelet Bases

Chapters 7-9 represent the heart of the book. Chapter 7 contains an expo- sition of the general notion of a multiresolution analysis (JVIRA) together with several examples. Next, we describe the recipe that gives the construc- tion of a wavelet basis from an l\IRA, and then construct corresponding examples of wavelet orthonormal bases. Chapter 8 describes the passage from the continuous domain to the discrete domain. First, properties of lVIRA are then used to motivate and define the quadrature mirror filter (QJVIF) conditions that any orthonormal wavelet filter must satisfy. Then the discrete wavelet transform (DWT) is defined for infinite signals. pe- riodic signals, and for finite sets of data. Finally the techniques used to pass from discrete filters satisfying the Ql\IF conditions to continuously defined wavelet functions are described. Chapter 9 presents the construc- tion of compactly supported orthornomal wavelet bases due to Daubechies. Daubechies's approach is motivated by a lengthy discussion of the impor- tance of vanishing moments in the design of wavelet filters.

Other Wavelet Constructions

Chapters 10 and 11 contain a discussion of two important variations on the theme of the construction of orthonormal wavelet bases. The first. in Chapter 10, shows what happens when you consider nonorthogonal wavelet systems. This chapter contains a discussion of Riesz bases, and describes the semi-orthogonal wavelets of Chui and Wang, as well as the notion of dual JVIRA and the fully biorthogonal wavelets of Daubechies, Cohen. and Feauveau. Chapter 11 discusses wavelet packets. another natural variation on orthnormal wavelet bases. The motivation here is to consider what hap- pens to the DWT when the "full wavelet tree" is computed. \Vavelet packet functions are described, their time and frequency localiiation properties are discussed, and necessary and sufficient conditions are given under which a collection of scaled and shifted wavelet packets constitutes an orthonormal basis on R. Finally, the notion of a best basis is described. and the so-called best basis algorithm (due to Coifman and Wickerhauser) is given.

xvi Preface

Applications Many wavelet books have been written emphasizing applications of the theory, most notably, Strang and Nguyen, Wavelets and Filter Banks, and Mallat's comprehensive, A Wavelet Tour of Signal Processing. The book by Wickerhauser, Applied Wavelet Analysis from Theory to Software, also contains descriptions of several applications. The reader is encouraged to consult these texts and the references therein to learn more about wavelet applications. The description of applications in this book is limited to a brief de- scription of two fundamental examples of wavelet applications. The first, described in Chapter 12, is to image compression. The basic components of a transform image coder as well as how wavelets fit into this picture are described. Chapter 13 describes the Beylkin-Coifman-Rokhlin (BCR) algo- rithm, which is useful for numerically estimating certain integral operators known as singular integral operators. The algorithm is very effective and uses the same basic properties of wavelets that make them useful for image compression. Several examples of singular integral operators arising in ordi- nary differential equations, complex variable theory, and image processing are given before the BCR algorithm is described.

Acknowledgments

I want to express my thanks to the many folks who made this book possible. First and foremost, I want to thank my advisor and friend John Benedetto for encouraging me to take on this project and for graciously agreeing to publish it in his book series. Thanks also to Wayne Yuhasz, Lauren Schultz, Louise Farkas, and Shoshanna Grossman at Birkhauser for their advice and support. I want to thank Margaret Mitchell for LaTeX advice and Jim Houston and Clovis. L. Tondo for modifying some of the figures to make them more readable. All of the figures in this book were created by me using MATLAB and the Wavelet ToolBox. Thanks to the MathWorks for creating such superior products. I would like also to thank the National Science Foundation for its support and to the George Mason University Mathematics Department (especially Bob Sachs) for their constant encouragement. I also want to thank the students in my wavelets course who were guinea pigs for an early version of this text and who provided valuable feedback on organization and found numerous typos in the text. Thanks to Ben Crain, James Holdener, Amin Jazaeri, Jim Kelliher, Sami Nefissi, Matt Parker, and Jim Timper. I also want to thank Bill Heller, Joe Lakey, and Paul Salamonowicz for their careful reading of the text and their useful comments. Special thanks go to David Weiland for his willingness to use the manuscript in an undergraduate course at Swarthmore College. The book is all the better

Preface xvii

for his insights, and those of the unnamed students in the class. I want give special thanks to my Dad, with whom I had many conversa- tions about book-writing. He passed away suddenly while this book was in production and never saw the finished product. He was pleased and proud to have another published author in the family. He is greatly missed. Finally, I want to thank my wife Megan for her constant love and support, and my delightful children John and Genna who will someday read their names here and wonder how their old man actually did it.

Fairfax, Virginia

David F. Walnut

Albrecht Durer (1471-1528), Melencholia I (engraving). Courtesy of the Fogg Art Museum, Harvard University Art Museums, Gift of William Gray from the collection of Francis Calley Gray. Photograph by Rick Stafford, @President and Fellows of Harvard College. A detail of this engraving, a portion of the magic square, is used as the sample image in 22 figures in this book. The file processed is a portion of the image file detail.mat packaged with MATLAB version 5.0.

Part I Preliminaries

Chapter 1

Functions and Convergence

1.1 Functions

1.1.1 Bounded (L (0) Functions

Definition 1.1. A piecewise continuous function f(x) defined on an interval I is bounded (or L =» on I if there is a number AI > 0 such that If (.1:) I :::; AI for all x E I. The L => -norm of a function f (J;) is defined by

Ilfll= = sup{lf(x)I::r E I}.

(1.1)

Example 1.2. (a) If I is a closed. finite intervaL then any function f (1;) continuous on I is also L00 on I (Theorem A.3). (b) The function f (;r;) = 1/ x is continuous and has a finite value at each point of the interval (0,1] but is not bounded on (0,1] (Figure 1.1). (c) The functions f(:1;) = sin(x) and f(x) = cos(;r;) are L00 on R. Also. the complex-valued function f(x) = e i .1' is L00 on R. In fact, II sin 11= = II cos II = = II e i:r II = = 1.

(d) Any polynomial function p(x) is not L00 on R but is L= on every finite subinterval of R.

(e) Any piecewise continuous function with only jump discontinuitiet-: is L= on any finite interval I.

1.1.2 Integrable (L 1 ) Functions

Definition 1.3. A piecewise continuous function f(x) defined on an interval I is integrable (or of class L 1 or simply L 1) on I if the integral

! If ( x ) 1 dx

is finite. The L1-norm of a function f(x) is defined by

II fill = ! If (x) I dx.

(1.2)

4 Chapter 1. Functions and Convergence

100 I I I I I I I I I I I I I I I I I I 90 --r--r--i--i-I--1--1--1--1-- 80 --

--

--

--

-

--

--

--

--+-- I I I I I I I I I 70 --t--t--

--

-

--

--i--i--+-- I I I I , I I I I 60 -+--t--t--t-

--

--i--i--i-- I , , , I I I I I 50 -t--t--t--t-l--1--i--1--f-- I I I I I I I I I 40 -t--t--r--r-1--1--1--1--t-- I I I I I I I I I 30 -t--t--r--r-1--l--i--f--f-- I I I I I I I I I

--

--

--

-

--

--

--4--

-- I I I I I I , I I I I I I I , I I I 10 --,--

--i--i-l--i--i--i--t-- I I I I 00 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-1

4

0 2 468

FIGURE 1.1. Left: f(x) = l/x is finite-valued but unbounded on (0,1]. Right: sin(x) (solid) and cos(x) (dashed) are L'= on R.

Example 1.4. (a) If f(x) is L= on a finite interval I, then f(x) is LIon I.

(b) Any function continuous on a finite closed interval I is LIon I. This is because such a function must be L= on I (Theorem A.3). (c) Any function piecewise continuous with only jump discontinuities on a finite closed interval I is LIon I. (d) For any 0 < a < 1, the function f(x) = lxi-a is LIon the interval [-1, 1]. Clearly f (x) is piecewise continuous with an infinite discontinuity at x = O. Thus the integral J

1 If(x)1 dx is improper and must be evaluated as an improper integral as follows:

jl,lxl-a dx

j -E 1 1 lim lxi-a dx + lim lxi-a dx E-+O -1 E-+O E 1 1 1 2 2 lim x-a dx = 2 - lim(l- El-a) = _. E-+O E 1 - a E-+O 1 - a

The above example shows that an L 1 function need not be L=. If a 2': 1, then f(x) is not LIon [-1,1]. (e) If a > 1, the function f(x) = x-a is LIon the interval [1, (0) since the improper Riemann integral

1= x-a dx

converges.

1.1. Functions 5

(f) If 0 :s; 0: :s; 1, then J(x) = x-a is not LIon [1,00). But J(x) is D XJ on [1, 00 ). This shows that an L CXJ function need not be LIon I if I is infinite. (g) The function J(x) = e- 1xl is integrable on R since the improper Rie- mann integral

f: e- Ixl d:£ converges. In fact, J

CXJ e- 1xl dx = 2.

We present below our first approximation theorem. It says that any func- tion LIon R can be approximated arbitrarily closely in the sense of the L 1 -norm by a function with compact support. Theorem 1.5 is illustrated in Figure 1.2.

Theorem 1.5. Let f(x) be £1 on R, and let E > 0 be given. Then there exists a number R such that if

{ f(x) g(x) = f(x)X[-R,R](x) = 0

if x E [-R, R], if x rf- [-R, R].

Then

f: If(x) - g(x)1 dx = Ilf - glh < E.

Proof: Since J (x) is integrable, the definition of the improper Riemann integral implies that there is a number v such that r

L If(r)1 d:£ = v

f: If(:£)1 d:£

Hence, given E > 0, there is a number TO > 0 such that if T 2': TO, then

IL: If(r)1 d:£ - L If(r)1 d:£1 = IL If(r)1 dr - vi < t. (1.3)

Pick a number R 2': TO, and define

g(x) = J(x) X[-R,Rj(X).

Then,

L: IJ(x) - g(x)1 dx

lim j r I J ( x) - g ( x ) I dx r -t CXJ -r lim r IJ(x)1 dx r-tCXJ J R<lx!<r IL: If(:£)1 d:£ - L: If(:£)1 drl < t

by (1.3).

D

6 Chapter 1. Functions and Convergence

1.5 1.5

1 -- --- -- --

0.5 - - -; - - - :- - - -: - . , , , , , , , , , , , , , , , , , , , , , , ,

, , , - , - - - ,- - - 1 - - -

0.5 - - - - - - - -

o 20 15 1 0 5 0 5 1 0 15 20

o 20 15 10 5 0 5 10 15 20

FIGURE 1.2. Illustration of Theorem 1.5. Left: Graph of f(x). Area of shaded region is < f. Right: Graph of g(x) with R = 10.

1.1.8 Square Integrable (L 2 ) Functions

Definition 1.6. A piecewise continuousfunction f(x) defined on an interval I is square-integrable (or of class £2 or simply £2) on I if the integral

l lf (X)1 2 dx

is finite. The £ 2 - norm of a function f (x) is defined by

( ) 1/2 IIfl12 = l lf (X)1 2 dx

(1.4)

Example 1.7. (a) Any function bounded on a finite interval I is also L 2 on I. This includes functions continuous on closed intervals and functions piecewise continuous on closed intervals with only jump discontinuities. (b) Any function that is VXJ and L 1 on any interval I (finite or infinite) is also £2 on I. (c) For any 0 < 0: < 1/2, the function f(x) = x-a is L 2 on the interval [-1,1]. Therefore an integrable function need not be bounded. If 0: 2: 1/2, then the corresponding f (x) is not L 2 on [-1, 1]. (d) If 0: > 1/2, the function f(x) = x-a is L 2 on the interval [1,00). If o :s; 0: :s; 1/2, then the corresponding f(x) is not L 2 on [1,00).

Theorem 1.8. (Cauchy-Schwarz Inequality) Let f(x) and g(x) be £2 on the

1.1. Functions 7

interval I. Then

If J(r) g(r) dxl S (f IJ(x)I' dr f' (f Ig(x)I' dr f' (1.5)

Proof: Let us assume first that J(x) and g(x) are real-valued, and let t be an arbitrary real number. Then

o :S; ! (J ( x) + t g ( x ) ) 2 dx = ! 1 J ( x ) 1 2 + 2t J ( x ) g ( x) + t 2 1 g ( x ) 1 2 dx t 2 ! Ig(x)12 dx + 2t ! J(x) g(x) dx + ! IJ(x)12 dx At 2 + Bt + C.

This expression represents a real-valued quadratic function of t that is nonnegative for all t. Hence its discriminant must be nonpositive (the dis- criminant of a quadratic expression at 2 + bt + c is b 2 - 4ac). Therefore,

2 4 (f J(x) g(x) dr) S 4 f If(x )1 2 dx f Ig(.'!;) 1 2 dr

and (1.5) follows. If J(;r) and g(x) are not real-valued, then we observe

If f(x) g(x) dxl s .I If(r)1 Iq(r)1 dr

(Theorem A.4) and then proceed as in the real-valued case.

D

From the Cauchy-Schwarz inequality, we can say something about the relation between L 1 and L 2 functions.

Theorem 1.9. Let I be a finite interval. If f(x) is L 2 on I, then it is L 1 on I.

Proof: By the Cauchy-Schwarz inequality,

! IJ(x)1 dx = ! 1 . IJ(x)1 dx s (f 1 dX) '/2 (f If(r)1 2 ) '/2 ( ) 1/2 1111/2 ! IJ(x)12

Since I is a finite interval and since J(x) is L 2 , the right side and hence the left side of the above inequality are finite. D

8 Chapter 1. Functions and Convergence

Remark 1.10. (a) The conclusion of Theorem 1.9 does not hold if I is an infinite interval. For example, let f (:r) = { 1

:r

:

:. Then f (:r) is £2 on R but is not LIon R.

(b) The converse of the Theorem 1.9 is false for both finite and infinite intervals. This means that if f (x) is LIon I, then it is not necessarily true that f(x) is L 2 on I. For example, let f(x) = X- 1 / 2 for x E (0,1). Then f(x) is L 1 but not L 2 on (0,1).

The following consequence of the Cauchy-Schwarz inequality allows us to conclude that linear combinations of £2 functions are also L 2 (Exer- cise 1.19). It is also known as the triangle inequality for L 2 functions; that is, it says that IIf + gl12 :s; IIfl12 + Ilglb (see Exercise 1.18).

Theorem 1.11. (Minkowski's Inequality) Let f(x) and g(x) be L 2 on the interval I. Then

( ) 1/2 ) ]/2 ( ) 1/2 flf(y) + g(x)I' d.T ::; (flf(y)I' d.T + flg(y)I'dr

(1.6)

Proof: By the Cauchy-Schwarz inequality,

1 1 f ( x) + g ( x ) 1 2 d:r: i If(:r W dx + 1 (1(:r) g(x) + f(x) g(x)) d:r + 1 Ig (:r) 1 2 dx < l lf (:r)1 2 d:r + 2 (/ If(:rW d:r) 1/2 (1 Io9 (:r)1 2 d:r) '/2 + 1 Io9 (:r)1 2 d:r ( (f If(:r)1 2 d:r) '/2 + (f lo9(:r)12 d:r) 1/2) 2

and (1.6) follows.

D

The following approximation theorem says that a function f (x) that is L 2 on R can be approximated arbitrarily closely by a compactly supported function in the sense of the L 2 -norm. The proof is very similar to that of Theorem 1.5 and is left as an exercise (Exercise 1.20).

Theorem 1.12. Let f(x) be L 2 on R, and let E > 0 be given. Then there exists a number R such that if g(x) = f(x) X[-R,R](X), then

i: If(x) - g(x)12 dx = Ilf - gll

< E.

1.1. Functions 9

1.1.4 Differentiable (en) Functions

Definition 1.13. Given n EN, we say that a function f (x) defined on an interval I is en on I if it is n- times continuously differentiable on I. CO on I means that f(x) is continuous on I. f(x) is Coo on I if it is en on I for every nE N. We say that f (x) is e;: on I if it is en on I and compactly supported, e

on I if it is CO on I and compactly supported, and e;:o on I if it is Coo on I and compactly supported.

Example 1.14. (a) If p(x) is a polynomial, then p(x) is C= on R. (b) The "hat" or "tent" function defined by J(x) = (1 -Ix I) X[-l,lj(X) is continuous but not differentiable on R since the derivative fails to exist at x = -1, 0, 1. Thus J(x) is CO (in fact C

) but not CIon R (Figure 1.3(a)). (c) We can generate an example of a smoother function by taking the tent function J(x) defined in (b) and taking its antiderivative. That is, we can define the function g( x) by

g(x) = I: J(t) dt.

Putting -00 as the lower limit of integration is just a notational convenience since we could just as easily have started the integration at any constant less than -1. By the Fundamental Theorem of Calculus (Theorem A.5), g' (x) = J (x) for all x E R since J (x) is continuous on R. Therefore g (x) is C 1 but not C 2 . Note that g(x) is not C

on R. (Figure 1.3(b)). (d) The function g(x) defined in (c) is L= on R but is not L 1 or £2 on R. In fact, since g(x) = 1 for all x > 1, g(x) does not even go to zero as x goes to 00. However, by modifying the construction in (c), it is possible to define a function that is C}. but not C 2 on R. In particular, such a function would be L 1 and £2 on R. The idea is to form two shifts of J(x), and subtract one from the other so that the resulting sum has integral zero. Then the anti.gerivative will vanish after a certain point. Specifically, define the function J (x) by

J(x) = J(x + 1/2) - J(x - 1/2)

and the function g(x) by

g(r) = L: 1(t) dt.

Then g(x) is C 1 but not C 2 for the same reason that g(x) is, and g(x) is CJ since it is supported in [-3/2,3/2] (Figures 1.3(c) and (d)).

10 Chapter 1. Functions and Convergence

(e) It is possible to define a sequence of functions with increasing smooth- ness based on the procedure outlined in (d). First define the function BO(l:) by Bo(x) = X[-1/2.1/2](X), and for each n E N. define Bn(:J:) by

f "l' / ,1'+1/2 Bn(x) = (Bn-l(t+l/2)-Bn_1(t-l/2))dt= B Il - 1 (t)dt. -(X) .1'-1/2

Note that Bl(X) is exactly the tent function defined in (b). Each Bn(;r) vanishes outside the interval [- ('lI + 1) /2. (n + 1) /2] and for n 2': 1 is e n - 1 but not en on R (Bo(.r) is not CO). The function BIl(J') is called the B- spline of order- n and is in fact a piecewise polynomial. 1\lore interesting properties of spline functions are given in Section 7.7.1.

08

06

04

02

-2 -1 5 -1 -0 5 a as 1 1 5 2 -2 -1 5 -1 -0 5 a as 1 1 5 2

1 08 06 04 02 a -0 2 -0 4 .06 -08

-15 -1 -as a as 1 2 25

-2 -1 5 -1 -0 5 a a 5 1

FIGURE 1.3. Top left: Graph of "hat" function f(:1') = (I-Ixl) X[-l,l](X). Top right: Antiderivative of f(x). Bottom left: Graph of f(x + 1/2) and - f(:1' - 1/2). Bottom right: Antiderivative of f(J')' This function is C/ on R but not C'2.

Exercises

Exercise 1.15. Prove each of the statements made in Example 1.2.

1.2. Convergence of Sequences of Functions 11

Exercise 1.16. Prove each of the statements made in Example 1.4.

Exercise 1.17. Prove each of the statements made in Example 1.7.

Exercise 1.18. Prove that each of the norms II. lip, for p = 1, 2, 00 satisfy the norm axioms: Given f(x) and g(x) LP on an interval I:

(a) II flip 2': O. (b) II flip = 0 if and only if f(x) == O. 1 (c) II Ct flip = I Ct III f II p, for every Ct E C. (d) Ilf + gllp :s; II flip + Ilgllp (this is known as the triangle inequality).

Exercise 1.19. Prove that the collection of functions LP on an interval I (p = 1, 2, (0) is closed under the formation of linear combinations. That is, show that if fn(x) is LP on I for 1 :s; n :::; N and if Ctl, ()i2, ..., C

N are complex numbers, then 2...:

=1 Ct n fn(x) is also LP on I.

Exercise 1.20. Prove Theorem 1.12.

1.2 Convergence of Sequences of Functions

1.2.1 N'umerical Convergence

Definition 1. 21. The sequence {an} nEN converges to the number a if for every E > 0, there is an N > 0 such that ifn 2': N, then Ian - al < E. In this case, we write an --+ a as n --+ 00, or lirn n -+= an = a. A numerical series, denoted 2...:

=1 an, converges to a number S if the sequence of partial sums {SN} NEN defined by SN = 2...::=1 an converges to S. In this case, we write 2...:

=1 an = S . We will frequently denote the series 2...:

=1 an by 2...:nEN an. A series 2...:nEN an converges absolutely if 2...:nEN Ian I converges.

Remark 1.22. (a) A fundamental property of the real numbers is known as the completeness property. The completeness property for the real num- bers says that every set of real numbers bounded above has a supremum (or least upper bound). (b) A sequence of real numbers {an}nEN is an increasing sequence if an :::; a n +l for all n E N. The completeness property of the real numbers implies

lSince we assume that J(a) is undefined if J(x) has a discontinuity at x = a, then "J(x) == 0" means "J(x) = 0 at each point of continuity of J(x)."

12 Chapter 1. Functions and Convergence

that any bounded, increasing sequence of real numbers always converges to its least upper bound. The partial sums of a series with nonnegative terms, (i.e., LnEN an, where an 2': 0) form an increasing sequence. Therefore, it follows that if a series of nonnegative terms is bounded, then it converges. (c) A sequence of real or complex numbers {an}nEN is Cauchy if for every E > 0, there is an N > 0 such that if n, m 2': N, then Ian - ami < E. Another consequence of the completeness property for real numbers is the following: Every Cauchy sequence of numbers converges.

Example 1.23. (a) Consider the series L

=o rn. The partial sum se- quence can be computed since

N

n SN = L r = n=O

1 - r N + 1 l-r

If Irl < 1, then SN -t 1/(1 - r) as N -t 00. Therefore, if Irl < 1, then

= 1

r n = _ L l-r' 71,=0

(b) Consider the series L

=11/n2. Clearly,

N 1 J: N 1 1

-< -dx=I--. L n 2 - x 2 N 71,=2 1

Therefore,

N 1 1 S N =

- < 2 - - < 2 L n 2 - N - , 71,=1

for all N. Since each of the terms 1/ n 2 is positive, {s N } N EN is a bounded, increasing sequence. Therefore, it converges to its least upper bound and the series L

=1 l/n2 converges. Note that we have proved that the series converges but we have made no statement about the value of its limit. The same argument can be used to show that the series L

=11/nP converges for every p > 1, but again does not give the value of its limit. ( c) The Weierstrass M - test is a well-known test for convergence of a series. Consider the series LnEN an. The Weierstrass M-test says that if Ian I :::; b n for all n and if LnEN b n converges, then LnEN an converges. For example, consider the series L

=1 cos(n)/n 2 . Since I cos(n)/n 2 1 :::; l/n2 for all nand since L

=11/n2 converges, so does L

=1 cos(n)jn 2 . Note that again we have proved that the series converges but have not given the value of its limit.

1.2. Convergence of Sequences of Functions 13

(d) A consequence of the Weierstrass M -test is the following. If a series converges absolutely, then the original series also converges. Absolute con- vergence is equivalent to saying that the series converges regardless of the order in which the terms are summed. It is not true that all convergent series are absolutely convergent. For example, it is shown in most calcu- 1 us books that the series 2::= 1 ( -1) r1 / n converges but the harmonic series 2::=1 l/n does not. (e) A doubly infinite sequence is a sequence of the form {an }nEZ, In dis- cussing the convergence of such sequences, we look at two limits, namely, lim n --+= a TI and lim n --+= a-n0 If both converge to the same number, say, to a, then we write liml nl--+= all = a. (f) A doubly infinite series is a series of the form 2::=-= an. In discussing the convergence of such series, we look at two series, namely, 2:

=1 an and 2:

=1 a- n . If both of these series converge, then there is no problem. If 2:

=1 an = S+ and 2:

=1 a_II = S-, then 2:

=-= an = ao+S+ +S- = S. In this case, we write also lilllN,M--+= 2::=

M all = S. We will frequently denote the series 2::=

= an by 2: nE z all or simply by 2:71 an. (g) If a doubly infinite series converges absolutely, then it converges regard- less of the order in which the terms are summed. This is not the case with se- ries that do not converge absolutely. Consider the series 2::=-= l/n, where the n = 0 term is understood to be zero. Clearly, this series does not con- verge absolutely. However, because of cancellation, SN = 2::=-N l/n = O. Hence, the symmetric partial sums converge to zero. However, if we define 2 'iN = 2::=-N l/n, then

N 2 S N :::: L 1/ x dx = In N --+ 00 as N --+ 00. Therefore, if a doubly infinite series does not converge abso- lutely, then the form of the partial SUIIlS must be given explicitly in order to discuss the convergence of the series. This is true of any series that converges but not absolutely.

1.2.2 Pointwise Convergence

Definition 1.24. A sequence of functions {fn (x) }nEN defined on an interval I converges pointwise to a function f (x) if for each Xo E I, the numerical sequence {fn(xo)}nEN converges to f(xo). We write fn (x) --+ f(x) pointwise on I, as n --+ 00. The series 2:

=1 fn(x) = f(x) pointwise on an interval I if fOT each Xo E I, 2:

=1 fn(xo) = f(xo).

Example 1.25. (a) Let !n(x) = x n , x E [0,1) for all n E N. Then !n(x) --+ 0 pointwise on [0,1) as n --+ 00. See Figure 1.4(a).

14 Chapter 1. Functions and Convergence

(b) Let

{ 2n:r in (.r) = 2 - 02nx

if X E [0.1/2n) if X E [1/2n, l/n) ifxE [l/n,l].

Then In (.1') --+ 0 pointwise on [0,1]. See Figure 1.4(b). ( c) Let

In (.r) = {

2n 2 x 2n - 2n 2 .'r o

if x E [0,1/2n) if x E [1/2n, l/n) if:r E [l/n, 1].

Then In (J;) --+ 0 pointwise on [0,1]. See Figure 1.4(c). ex:, 1 (d) The series

J.n = - pointwise on (-1.1). L 1 - :1' n=()

CXJ :1' n ( e) The series L -, = c. r pointwise on R. n. n=()

ex:, L cos n.r (f) The series _ 2 cOllverges pointwise on R to its limit by the n n=l Weierstrass AI-test.

()() cos n:1' (g) The series

- converges at odd multiples of 7T (since it reduces

n n=1 to the alternating series L::

=l(-l)nln) but diverges at even multiples of 7T (sincE' it reduces to the harmonic series). In fact. it can be shown that the series converges for all .r that are not even multiples of 7T.

1.2.3 Uniforrn (LOO) Convergence

Definition 1.26. The sequence {fn (X)}nEN converges uniformly on I to thE' function f(.r) if for every E > O. there is an N > 0 such that if n 2': N, then Ifn(x) - f(x)1 < F for all x E I. We writf' fn(x) ---+ f(x) uniformly on I as n ---+ oc. The series L::;:l fn(:r) = f(x) uniformly on I if the sequence of partial s'ums SN(X) = L:::=l fn(x) converges 1miformly to f(x) on I.

Remark 1.27. (a) With uniform convergence. for a given E the same N works for all x E I, whereas with pointwise convergence N may depend on both f and x. In other words, uniform convergence says that given E > 0 there is an N > 0 such that for all n 2': N, the maximum difference between In (x) and 1 (x) on I is smaller than f. Because of this. uniform convergence

1.2. Convergence of Sequences of Functions 15

02

08

06

04

o 01 02 03 04 05 06 07 08 09 1 0 01 02 03 04 05 06 07 08 09 1 8

I I 1\=2- j - . I I

04 05 06 07 08 09 1

FIGURE 1.4. Top Left: Graph of f7l(x) = .1''' on [0,1) for TI = 2,4.8. Top Right: Graph of f,,(:r;) on [0,1) where fT/(.D) is defined in Exam- ple 1.25(b). Bottom: Graph of f" (T) on [0.1) where f" (.1') is defined in Example 1.25(c).

is also calleel LX con:ueTgencc. That is, fn(.r) -t f(.r) uniformly on I if anel only if Ilfll - fllx -t 0 as n -t Xi. (b) In Example 1.25(b), thp convergpnce of fll (J') to 0 is pointwisp but not uniform. This is becausp the maximum elifferencp between fll (.1') and thp limit function f(J:) == () is 1 no matter ,vhat n is. In other words. Ilfll - flloo = Ilflllloo = 1 for all n, anel so Ilfll - fll= -1+ () as n -t Xi.

(c) In Example 1.25(c), the convergpnce of fll(.r) to 0 is also pointwisp but not uniform. In fact, in this casp, Ilfll - fll= = Ilfllllx = n for all lL Therefore Ilfll - fllx -t Xi as n -t Xi. In fact. therp are no examples of sequcnces that converge uniformly on an interval but not pointwise. In other words, t he following theorem holds.

Theorem 1.28. If f" (.1') --+ f(.r) in L = on an mtcTval I. then f,,(.1') --+ f(x) pointwise on I.

Proof: Exercise 1.44.

D

An important theorem from advanced calculus is the following. Its proof

16 Chapter 1. Functions and Convergence

is left as an exercise but can be found in almost any advanced calculus book (for example, Buck, p. 266, Theorem 3).

Theorem 1.29. If fn(x) -t f(x) uniformly on the interval I, and if each !n(X) is continuous on I, then f(x) is continuous on I.

Proof: Exercise 1.45.

D

Example 1.30. As an illustration of Theorem 1.29, let

{ -I fn(x) = nt

if x E [- 1, - 1/ n ) if x E [-I/n, l/n) if x E [1/n,l].

Then each fn(x) is continuous on [-1,1] and {fn(X)}nEN converges point- wise to the function .f (x) defined by

{ -I f(x) =

if x E [- 1, 0) if x = 0 if x E (0,1],

which has a jump discontinuity at x = 0 (see Figure 1.5). It can be shown directly that fn (x) does not converge to f (x) in L = on [-1, 1], but a different argument utilizing Theorem 1.29 would be as follows. If fn(:J;) -t f(,1:) in L= on [-1,1], then since each fn(x) is continuous, Theorem 1.29 implies that f (x) should also be continuous. Since this is not the case, the convergence cannot be in L=.

Example 1.31. (a) The sequence {Xn}nEN converges uniformly to zero on [-0:,0:] for all 0 < 0: < 1 but does not converge uniformly to zero on (-1,1).

= 1 (b) The series '" x n = - uniformly on [-0:,0:] for all 0 < 0: < 1, but L I-x n=O not on (-1, 1).

xn ( c) The series L , = eX uniformly on every finite interval I, but not on n. n=O

R.

= L cos nx (d) The series converges uniformly to its limit on R by the n 2 n=l Weierstrass 1\11- test.

1.2. Convergence of Sequences of Functions 17

1 0.8 0.6 0.4 0.2 o -0.2 -0.4 -0.6 -0.8 -1 -1

I I I I ------------- .. I I I I I I I I

------------- I I I , I I I ,

, " , " ------------- , , , I I , , ,

------------- , , I I , , I ,

, I I , .. .. ,_ .. _I.. .. .J .. .. l .. .. I I I , I , , , I , , I .. .. ,_ .. _'.. .. .J .. .. 1. .. .. , , , I

-n

2 :

.0.8 -0.6 -0.4 .0.2 0 0.2 0.4 0.6 0.8 1

1 0.8 0.6 0.4 0.2 o -0.2 -0.4 -0.6 -0.8 -1 -1

I I , I .. .. ,- .. -," .. "'; .. .. i .. .. I I , ,

" '" " I" ----............ ------- --.. ---.. , , , , , , 1 , , I I , I I , I , I I , I I , I , I , , I , , I ........ -------...... --- ------- I , , I I , I , I , , I , , I , I , , I , , I , , , I , I I , , ---- --.............. ----------- , , I , I I , I , I I , I I , I , , I , I , , I .. _,_ .. _I.. .. .J .. .. !. .. .. I.. .. _,_ .. .1.. .. .!. .. .. , I , I I , I I I I , , , , I , I , , I , , I , - -: - - -: - -

- -

- - '" - -: - -

- - ; - -

, '" I , I , , I , I I , I .. .. ,_ .. _,.. .. .J .. .. L .. .. ,_ .. _'.. .. _, .. ..

.. _ , I , I I , I I , I , , I , I , I I , I , , I , .. _,_ .. _'.. .. .J .. .. L .. .. ,_ .. _,_ .. oJ .. .. L .. .. I , I I , I , , I , , , I I , , , , I , I I , I .. .. ,_ .. _'.. .. .J .. .. L .. .. ,_ .. _,.. .. .J .. .. L .. _ I I I I I I I , , I , I I , I , I I , I , , I , .. .. '_ .. _, _ _ oJ _ _ I. _ _ ,_ _ _' _ _ .J _ _ L _ _ I , I , , , , I I , , , , , , , I , I , __'__.J__L__

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

FIGURE 1.5. Left: Graph of fn (x) of Example 1.30 for n Right: Graph of the limit function f(.r).

1.2.4 Mean (£1) Conver:qcnce

2,4,8.

Definition 1.32. The sequence {fn(X)}'tEN defined on an inter-val I converges in mean to the function f (:1;) on I if

lim J 'lfn (.1') - f(:1') I dx = O. n-+oo I

We write fn(x) ---+ f(:r) in mean on I as n ---+ 00. Mean convergence is also refer-red to as £1 convergence because fn(x) ---+ f(.r) in mean on I as n ---+ 00 is identical to the statement that lim n -+= IIf" - fill = O. The series L::1 fn (x) = f(.r) in mean on I if the sequence of par-tial sums sN(:r) = L

=l f,,(x) conver-ges in mean to f(x) on I.

Mean convergence can be interpreted as saying that the area between the curves y = fn (x) and y = f (x) goes to zero as 11 ----t oc. This type of convergence allows point values of fn(x) and f(x) to differ considerably but says that on average the functions In (x) and f(;r;) are close for large n.

Example 1.33. (a) Let In (x) = .r".:1' E [0,1) for all 71 E N. As we have seen in Example 1.25(a), this sequence converges to f(x) == a pointwise on [0,1) but not uniformly on [0,1). Since

1 1 / 1 1 Ilfn - fll1 = IIn(x) - f(x)1 dx = :I,n dx = - ----t a o .0 71+1

as rJ ----t 00, fn (x) ----t 0 in mean on [0.1).

18 Chapter 1. Functions and Convergence

(b) Consider the sequence {fn(X)}nEN defined in Example 1.25(b). The sequence converges pointwise but not uniformly to f(x) = 0 on [0,1]. Since the area under the graph of f n (x) is 1/2n for each n, the sequence also converges in mean to f(x) on [0,1]. In this example, we can see the character of mean convergence. If n is large, the function f n (x) is close to the limit function f(x) = 0 (in fact identical to it) on most of the interval [0,1], specifically on [l/n,l], and far away from it on the rest of the interval [O.l/n). However, on average, fn(x) is close to the limit function. (c) The sequence {fn(X)}nEN defined in Example 1.25(c) tells a different story. The sequence converges pointwise but not uniformly to f(x) = 0 on [0.1], but since the area under the graph of fn(x) is always 1, fn(x) does not converge to f(x) in mean. The width of the triangle under the graph of f(x) decreases to zero, but the height increases to infinity in such a way that the area of the triangle does not go to zero.

The above examples show that sometimes pointwise convergence and mean convergence go together and sometimes they do not. The proof of the following theorem is left as an exercise (Exercise 1.47).

Theorem 1.34. If fn(x)

f(x) in L= on a finite interval I, then !n(X)

f(x) in LIon I.

Remark 1.35. (a) The conclusion of Theorem 1.34 is false if the interval I is infinite. Consider for example the sequence fn(x) = (l/n)X[o.n](x). Then fn(x) --t 0 in L oo on R but J

Ifn (x) - 01 dx = 1 for all n, so that f n (x) does not converge to zero in L 1 . (b) The converse of Theorem 1.34 is also false, as can be seen by considering Example 1.33(b). In this example, fn (x) converges to 0 in LIon [0, 1] but does not converge to 0 in L= on [0,1]. (c) In all of the examples of mean convergence considered so far, the se- quences have also converged pointwise. l'vlust this always be the case? The answer turns out to be "no," as is illustrated by the following example.

Example 1.36. Define the interval Ij.k by Ij.k = [2- j k, 2- J (k + 1)), for j E Z+ and 0 ::s; k ::s; 2) - 1. Let us make some elementary observations about the intervals Ij,k'

(a) Each Ij.k is a subinterval of [0,1). (b) The length of Ij,k is 2- J ; that is, Ifj.kl = 2- j . (c) Each natural number n corresponds to a unique pair U, k), j E Z+ and 0 ::s; k ::s; 2 j - 1, such that n = 2 j + k. For each n E N, call this pair Un, k n ). As n --t 00, jn --t 00 also.

1.2. Convergence of Sequences of Functions 19

(d) For each j, the collection of intervals {Ij.k H

(/ forms a partition of [0, 1); that is, the intervals are disjoint and cover all of [0, 1).

Now, define fn(x) = X1Jn,k n (x). Then since IIjn,k n I --+ 0 as j --+ 00, f n (x) --+ 0 in mean on [0, 1). However, f n ( x) does not converge to zero pointwise because for every x E [0,1), there are infinitely many n for which fn(x) = 1. Therefore, fn(x) does not converge to anything at any point of [0,1). See Figure 1.6.

1 8 6 4 2 0

1_

00102030405060708091 00102030405060708091 00102030405060708091 00102030405060708091

_ '1--

-

01--

00102030405060708091 0010203040506070809 1 00102030405060708091 00102030405060708091

-

01----

00102030405060708091 00102030405060708091 00102030405060708091 0 01020304050607 0809 1

FIGURE 1.6. Graph of fn(x) of Example 1.36 for 1 :S n :S 12.

1.2.5

Mean-square (L 2 ) Convergence

Definition 1.37. The sequence {fn(X)}nEN converges in mean-square to the function f(x) on an interval I if

lim ! Ifn(x) - f(x)12 dx = O. n-+= I

We write fn(x) ---+ f(x) in mean-square on I as n ---+ 00. Mean-square convergence is also referred to as L 2 convergence because f n (x) ---+ f(x) in mean on I as n ---+ 00 is equivalent to the statement that lim n -+= Ilfn - fl12 = O.

20 Chapter 1. Functions and Convergence

The series L:=l fn(x) = f(x) in mean-square on I if the sequence of partial sums SN(X) = L:=l fn(x) converges in mean-square to f(x) on I.

The proof of the following theorem is left as an exercise (Exercise 1.48). The proof of the first part is similar to the proof of Theorem 1.34 and the proof of the second part makes use of the Cauchy-Schwarz inequality (Theorem 1.8).

Theorem 1.38. (a) If fn(:£) -+ f(x) in L= on a finite interval I, then fn(x) -+ f(x) in L 2 on I.

(b) If fn(x) -+ f(x) in L 2 on a finite interval I, then fn(x) -+ f(x) in LIon I.

Remark 1.39. (a) The conclusion of Theorem 1.38( a) is false if Loo con- vergence is replaced by pointwise convergence. Example 1.33( c) shows a sequence that converges to zero pointwise on [0,1] but not in LIon [0,1]. By Theorem 1.38(b), the sequence does not converge in L2 either, for if it did, then Theorem 1.38(b) would imply that it also converged in L 1 . (b) The conclusion of Theorem 1.38 (a) is false if I is an infinite interval. For example, if fn(x) = (l/y1n) X[O.n] (:r:), then fn(x) --+ 0 uniformly on R, but I

Ifn(:x;) - 01 2 dx = 1 for all n, so that f(:r) does not converge to 0 in L 2 .

(c) The conclusion of Theorem 1.38(b) is false if I is an infinite interval. For example, if fn(x) = (l/n) X[O.n](x), then since 1 00. 2 1 1 If n (;:r;) - 01 dx = 2" n = - --+ 0,

oo n n

as n --+ 00, fn(x) --+ 0 in L2 on R. but

I: If,,(x) - 01 dx = 1

for all n so that fn(x) does not converge to 0 in L 1 on R. (d) The converse of Theorem 1.38(a) is false. To see this, consider Exam- ple 1.33(b). In this example, it can be shown directly that !n(x) --+ 0 in L 2 on [0,1] (Exercise 1.49). However, !n(x) does not converge to 0 in L oo on [0,1]. (e) The converse of Theorem 1.38(b) is false. To see this, let fn (x) = ylnX(O,l/n](X). Then

(I 1 Jo Ifn(x) - 01 dx = Vii --+ 0

1.2. Convergence of Sequences of Functions 21

as n ----t 00 so that in (x) ----t 0 in £1 on (0,1]. However, 1 1 1fn (:r:) - 01 2 dx = 1 () for all n so that in (x.) does not converge to 0 in £2 on (0, 1]. (f) Finally, note that Example 1.36 shows that £2 convergence does not imply pointwise convergence since the sequence defined there also converges to 0 in £2 on [0,1).

1.2.6 Interchange of Limits and Integrals A problem that we will encounter frequently in this book is the following. Suppose that a sequence of functions {fn (x) } nEN on an interval I converges in some sense described in one of the previous four subsections to a function f(x) on I. Under what conditions is it true that lirn r in (.r:) d.L = r f(.r) d:z:? 11--+00 iI iI

Since we can write i (:z:) = lirnll--+= fll (.r:). the above can be rewritten as lim / . In (:z:) dx = / . lim ill (.r;) d:l', n--+oo. I . I 71--+00 and this problem is often stated as: When can we e:rchange the limit and the integral Y The most typical form in which this problem arises is when the sequence is a sequence of partial sums of a series of functions. In this case, the equivalcnt question is: When can we 'integrate a series of fnnct'ions teT'TT/,- by-term Y To see this, recall that the integral of aw finite sum of functions is the sum of the integrals. so that if SN(X:) = 2::71=1 ill (.z:), then

.f SN (x) dx = tf. J" (l) d.r = t.f J" (x) do;. I I n= 1 II = 1 I

If we could interchange the limit and the integral in this case, we would have

! f J,,(x) dx I n=1

r lirn SN(X) dx iI N --+ oo lirn .f s N ( x) dx N --7= I N lirn L r in(x) dx N--+oo 71=1 iI f f J,,(X) dx. 71=1 I

22 Chapter 1. Functions and Convergence

The following theorem gives several conditions under which interchanging the limit and the integral is permitted.

Theorem 1.40.

(a) If In (X) -+ f(x) in LIon I, then

lim ! Irt(X) dx = ! I(X) dx. 71-+= I J

(b) If In (x) -+ I (x) in L 00 on a finite inteT'val I, then

lim r.f 71 ( x) dx = r I ( x) dx. 71-+00 J J JI

(c) ff fn(x) -+ I(x) in L 2 on a finite intenJaI I, then

lim r In(x) dx = r f(x) dx. n-+ooJI J!

Proof: (a) Let f1/(x) --+ f(x) in LIon I. Then

lim I r fn(:E) dx - r f(x) dx l n--+=I./I ./1

lim I ! (fn (.I:) - f (.T )) d.T i n--+oo I < lim ! Ifn (x) - f(x)1 d:r n--+oo. I

O.

(b) By Theorem 1.34, if fn(x) --+ f(x) in L oo on I, then it also converges in L 1 . Then the result follows from part ( a). (c) By Theorem 1.38(b), if fn(x) --+ f(x) in L 2 on I, then it also converges in L 1 . Then the result follows from part (a). D If I is an infinite interval, then the conclusions of Theorems 1.40(b) and 1.40( c) are both false, as can be seen by considering the example given in Remark 1.39(c). In this example, f1/(x) --+ 0 both in L= and L 2 on [0, (0). However, since JI fn (x) dx = 1 for all n,

lim ! fn(x) dx = 1 71--+00 I

but

! lim fn(x) dx = ! Odx = O. I n--+oo I

However, in the case of infinite intervals. we can prove a useful theorem by making an additional assumption on the sequence {fn (x )}nEN.

1.2. Convergence of Sequences of Functions 23

Theorem 1.41. Suppose that for every R > 0, fn(x) ---+ f(x) in VXJ or in L 2 on [-R, R]. That is, for each R > 0, lim J R Ifn(x) - f(x)12 dx = 0, or lim sup{lfrt(x)- f(x)l: x E [-R, R]} = O. n-+oo -R n-+oo

If f(x) is LIon an interval I and if there is a function g(x), Lion I, such that for all x E I and all n E N, Ifn(x)1 :S g(x), then lim J fn(x) dx = J f(x) dx. (1.7) n-+oo I I

Proof: If I is a finite interval, then there is nothing to do by Theo- rem 1.40(b) and (c), so we may assume that I is infinite, and for con- venience we will take I = R. By Theorem 1.40(a), it will be sufficient to prove that fn(x) --+ f(x) in L 1 on R. Let E > O. Since f(x) and g(x) are L 1 on R, by Theorem 1.5, there is a number R > 0 such that

1 If(.T) I dx < E/3 Ixl>R

and

rig ( x ) I dx < f / 3. J 1 :1'1>R

Therefore, using the triangle inequality for the L 1 -norm (Exercise 1.18(c)),

Ilfn - fill = f 'R Ifn(.r) - f(x)1 dx + r Ifn(.r) - f(:1;) I dx -R J1xl>R < j R Ifn(x) - f(x)1 dx + 1 Ifn(x)1 dx + 1 If(x)1 dx -R Ixl>R l.rl>R < j R Ifn(x) - f(x)1 dx + I Ig(x)1 dx + 1 If(:1;)1 dx -R . l:rl>R l.rl>R < f: If,,(x) - f(x)1 dx + 2c/3. (1.8) By Theorem 1.34 and Theorem 1.38(b), if fn(x) --+ f(:r) in L oo or L 2 on [- R, R], then it also converges in L 1 on [- R, R]. That is, lim j R Ifn(x) - f(x)1 dx = O. n--+oo -R

Hence, there is an N such that if n 2': N, then

f: If,,(x) - f(x)1 dx < c/3.

Therefore, if n 2': N, then

IL: fn(x) dx - L: f(x) dxl-c; IIf" - fil, < c.

24 Chapter 1. Functions and Convergence

and (1. 7) follows.

D

Next we present a variant of Theorem 1.41.

Theorem 1.42. Suppose that for every R > 0, fn(x) ---+ f(x) in L= or in £2 on [- R, R]. If for every E > 0, there is an R > 0 and an N E N such that for all n

N,

r Ifn(x)1 dx < E. J1xl>R

Then

lim 1 00 fn(X) dx = 1 00 f(x) dx. n-+oo -00 -00

(1.9)

Proof: The proof is the same as that of Theorem 1.41, except that we choose R > 0 and N E N such that for all n 2: N,

J If(x)ldx < E/3 Ixl>R

and

J Ifn(x)ldx<E/3. Ixl>R

Then (1.8) becomes

R Ilfn - fill = J Ifn(x) - f(x)1 d;z; + I Ifn(x) - f(x)1 dx -R . IJ:I>R < l R Ifn(x) - f(x)1 dx + r Ifn(x)1 dx + r If(x)1 d:r -R J1;:cI>R J1x'I>R < J R Ifn(x)-f(x)ldx+2E/3 (1.10) -R

from which (1.9) follows.

D

Exercises

Exercise 1.43. Prove each of the statements made in Example 1.25.

Exercise 1.44. Prove Theorem 1.28.

Exercise 1.45. Prove Theorem 1.29.

Exercise 1.46. Prove each of the claims made in Example 1.31.

Exercise 1.47. Prove Theorem 1.34.

Exercise 1.48. Prove Theorem 1.38.

1.2. Convergence of Sequences of Functions 25

Exercise 1.49. Prove that if fn(x) is defined as in Example 1.33(b), then fn(x) -+ 0 in L 2 on [0,1].

Exercise 1.50. (a) A sequence of functions {fn(X)}nEN defined on an interval I is said to be uniformly Cauchy on I if for every E > 0, there is an N > 0 such that if n, m 2:: N then Ilfn - frnll= < E. Prove that any sequence that converges in L= on I is uniformly Cauchy on I. (b) A sequence of functions {fn(X)}nEN defined on an interval I is said to be L 1 Cauchy on I if for every E > 0, there is an N > 0 such that if n, m 2:: N, then Ilfn - frnl11 < E for all x E I. Prove that any sequence that converges in L 1 on I is L1 Cauchy on I. (c) A sequence of functions {fn(X)}nEN defined on an interval I is said to be L2 Cauchy on I if for every E > 0, there is an N > 0 such that if n, m 2:: N, then Ilfn - frnl12 < E for all x E I. Prove that any sequence that converges in L2 on I is L 2 Cauchy on I.

Chapter 2

Fourier Series

2.1 Trigonometric Series

2.1.1 Periodic Functions

Definition 2.1. A function f(x) defined on R has period p > 0 if f(x + p) = f(x) for all x E R. Such a function is said to be periodic.

Remark 2.2. (a) The functions sin(x) and cos(x) have period 27T. The functions sin( ax) and cos( a.1:) , a > 0, have period 27T / a. (b) If f(x) has period p > O. it also has period kp. for kEN. Hence a periodic function can have many periods. Typically the smallest period of f (x) is referred to as the period of f ( x ) .

Definition 2.3. Given a function J(x) on R. and a number p > O. the p- periodization of J(x) is defined as the function

Jp(x) = L f(x + np). nEZ

(2.1)

pTOvzded that the sum makes sense. See Figure 2.1.

Remark 2.4. (a) It is easy to verify that in fact the function fp(x) has period p by making a change of summation index in the sum on the right side of (2.1). Specifically, fp(:J: + p) = L f(x + p + np) = L f(.1: + (n + l)p) = L f(.J: + np), nEZ TlEZ nEZ

where we have made the change of summation index n t---+ n + 1. (b) If f(x) is compactly supported, then the sum in (2.1) will converge pointwise on R. This is because for each x the sum will have only finitely many terms.

(c) If f (x) is supported in an interval I of length p, then fp (:1') is referred to as the peTiod p extension of f(x). This is because for x E I. fp(x) = LnEZ f(x + np) = f(x) since all terms in the sum besides the n = 0 term are zero. (Why?) Another way of thinking of this is that we are taking infinitely many copies of the function f (x) and placing them side-by-side on the real line.

28 Chapter 2. Fourier Series

o -2 -15 1 -05

,---r \Ur I I " I I 25 ,- tit, --I n t I , I, I 2

n

---lLl-n: , ! ': : , I , I' I 15 r1----j+-+- , : " :' : r'r

25

15

05

05

05

15

05 1 1.5 2

05

o -2 -1.5 1 -05

05

15

FIGURE 2.1. Top Left: Graph of f(x). Top Right: Graphs of f(x + np) for -2 ::; n ::; 2 and p = 1. Bottom: Graph of the I-periodization of f(x).

2.1.2

The Trigonometric System,

Definition 2.5. Given a > 0, the collection of functions

{e27rlnr/a}nEZ

(2.2)

is called the (period a) trigonometric system.

Remark 2.6. (a) Recall Euler's formula: e i . r = cos(x) + i sin(x). This formula can be proved by expanding both sides of the equation in a Taylor series (Exercise 2.20). Therefore e27rinx/a = cos(21Tnxja) + i sin(21Tnxja),

and it follows from this that each element in the trigonometric system has period a. (b) The period a trigonometric system is sometimes given in the form

{I, cos(21Tnx j a), sin(21Tnx j a)}TI EN.

(2.3)

Systems (2.2) and (2.3) can be obtained from each other by forming simple

2.1. Trigonometric Series 29

linear combinations. Specifically, for nEZ, c2

inxja = { cos(21f'nx/a)

i sin(2;mx/a) and for n EN,

if n i- 0, if n = 0,

e27rtTlx/a + e- 27rin :x;/a cos(27Tnx j a) =

2

and

e27rinx/a _ e-27rinx/a sin(27Tnxja) = ./ 2'L (c) A function that can be written as a finite linear combination of elements of the (period a) trigonometric system is called a (period a) trigonometric polynomial. That is, a trigonometric polynomial has the form N f(x) = L c(n) e27rtnx/a TI = Al

for some lvI, NEZ and some coefficients c( n).

Theorem 2.7. The period a trigonometric system (2.2) satisfies the following orthogonality relations:

1" c2nnxl" e -2nmxl" dx

{ :

if n -I m, if n = m.

(2.4)

Proof: Exercise 2.22.

D

Remark 2.8. Note that since the functions e27rinx /a all have period a. the integral in (2.4) can be taken over any interval of length a. For example,

j a/2 e27rtnx/a e-27rirnx/a dx -a/2

1" c2nin.xja c- 2nim "ja dx

{

if n i- m, if n = m.

A fundamental problem in Fourier series is the following: Given a func- tion f (x) with period a > O. can we write

f(x) = L c(n) e27rinx/a nEZ

(2.5)

for some choice of coefficients {c( n) }nEZ ? This problem leads to three related questions that will be answered in the following subsections:

30 Chapter 2. Fourier Series

(a) In order fOT (2.5) to hold, what must the coefficients c(n) be? (b) Assuming we know the answer to question (a), in what sense does the series on the right side of (2.5) converge? (c) As.mming we know the answers to questions (a) and (b), does the se- ries on the right of (2.5) converge to f(.r;), or to some other funct'ion?

2.1.3 The Fourier Coefficients

Let us begin by answering question (a) above.

Definition 2.9. Given a function f(x) with period a, the Fourier coefficients of f (x) are defined by

1 1 a c(n) = - f(x) a 0

dx

for

n E Z.

(2.6)

provided that those integrals make sense. For example. if f (x) is L 1 on [0. a], then the integral in (2.6) converges for each n.

Remark 2.10. The definition of the Fourier coefficients of a function f(x) is by no means arbitrary. In fact we are essentially forced to define them that way by the following argument. Suppose that in fact f(x) = LnEZ c(n) e27rinxja. Then in light of Theo- rem 2.7. for Tn E Z fixed,

1 0 o f (x) e -27rimxj a &1'

1 a L c(n) e27rin:rjo e-27rinu'/0 dx o nEZ L c(n) 1 a e27rwxja e- 27rlTn :rja dx = a c(rn), nEZ

since by (2.4), the only nonzero term in the sum is the n = m term. Note that the above argument is not a rigorous proof since we interchanged an integral and an infinite sum without having any idea how or even if the sum converged. However. the argument is sufficient motivation for defining the Fourier coefficients as in Definition 2.9.

Definition 2.11. Given a function f(x) with period a, L 1 on [0, a], the Fourier series associated with f(x) is defined as the formal series

'"'" ( ) 27rtrJ.r/a LC n e , nEZ

(2.7)

where the c(n) are defined by (2.6). We refer to (2.7) as a (

formal series" since we do not yet know how or if the series converges. We write

f( ) "'-' '"'" ( ) 27rln.r/a X LC n e .

nEZ

2.1. Trigonometric Series 31

Remark 2.12. It is possible to rewrite the Fourier series of a function in terms of the real trigonometric system defined by (2.3). To see this, note that

L c(n) e27rinx/a nEZ

'"'" ( ( 2 7rnX ) ( 2 7rnX ) ) c(O) + L c( n) cos -----;;- + i sin -----;;- nEN '"'" ( ( 2 7rnx ) ( 2 7rnx ) ) + L c( -n) cos --;;- - i sin --;;- nEN '"'" ( 27rnx ) c(O) + L (c(n) + c( -n)) cos -----;;- nEN '"'" ( 27rnx ) + L i (c(n) - c( -n)) sin --;;- . nEN

Conversely, a series of the form

Ao + L An cos(27rnx/a) + Bn sin(27rnx/a) nEN

can be rewritten as

L c(n) e27rinx/a, nEZ

where

c(O) = Ao,

( ) An - i Bn en = 2 '

n > 0,

( ) A-n + i B-n c n = 2 '

n < O.

Example 2.13. (a) Let f(x) be the period 2 extension of the function X[-1/2,1/2](X). The Fourier coefficients of f(x) are

( ) 1 / 1 X ( ) -27rtT1x/2 d c n "2 -1 [-1/2,1/2] X e x 1 / 1/2 . - e- 7rtT1X dx 2 -1/2

[e-7rin/2 _ e 7rin / 2 ] 2 7rzn 1 2 - - sin( 7rn/2) 2 7rn ! 0 if n is even, n #- 0,

(_1)(n-1)/2 ifn is odd, 7rn 1 if n = O. 2

32 Chapter 2. Fourier Series

The Fourier series associated to f (x) is

f(x)

(X)

+ ""'" 1 (_l)k e7fi(2k+l)X 2 L TI(2k + 1) k=-(X) 122 - + - cos(Tlx) - - cos(3T1x) + 2 TI 3T1

See Figures 2.2 and 2.3. (b) Let f(x) be the period TI extension of the function x X(O,7f)(x). Then

c(O) = (1/T1) fo 7f X dx = TI,

and for n #- 0,

1 1 7f . c(n) = - x e- 2inx dx =

. TI 0 2n

Therefore,

( i ""'" 1 2inx ""'" sin(2nx) f x) '"'-' TI + - L - e = TI - L . 2 n n nEZ nEN

See Figure 2.4. (c) Let f(x) be the period TI extension of the function x XC-7f/2,7f/2) (x). Then c(O) = 0, and for n #- 0, c( n) = (-l)n iTi 12n, so that

f(x) '"'-' i ""'" (-l)n e 2inx = _ ""'" (-It sin(2nx) . 2 L n L n nEZ nEN

(d) Let f(x) be the period 2T1 extension of the function Ixl X(-7f,7f)(x). Then

TI 4 ""'" 1 f(x) '"'-' - - - L ( )2 cos((2n + l)x). 2 TI 2n + 1 nEN

2.1.4 Convergence of Fourier Series

Definition 2.14. A function f (x) on a finite interval I is piecewise differen- tiable on I if (a) f(x) is piecewise continuous on I with only jump discontinuities (if any), (b) l' (x) exists at all but finitely many points in I and (c) l' (x) is piece- wise continuous on I with only jump discontinuities (if any). A function f (x) is piecewise differentiable on an infinite interval I if it is piecewise differentiable on every finite subinterval of I.

-2 -15 -1 -05 0.5 15 2 .15 -10 -5 25 1.5 05 0 -3 -2 -1 0 2 3 4 5 -15 .10 .5

I I I I I n_ i _n I I I I i I I

0.8

0.6

04

02

2.1. Trigonometric Series 33

05 nuln n\nm I _m

-- _Ln_ I I I I I I I I I I I 04 - n-i-----t- nn _____

n --

----- I I I I 03 -----

--n-t---- nn

--n+-n- I I I I I I I I 02 --- +----t ----1--- n 1 n --- I I I I I I I I 0.1 ---- t-----t -n-1- m +----

: .'=tTb

LIbjr'" . I I I I I

10 15

10 15

FIGURE 2.2. Top left: Graph of f(x) from Example 2.13(a). Top right: Graph of Fourier coefficients of f(x). Bottom left: Graph of f(x) from Example 2.13(b). Bottom right: Graph of absolute value of Fourier coefficients of f(x).

Example 2.15. (a) Any function 0 1 on I is also piecewise differentiable on I.

(b) If I is any finite interval, then the function XI (x) is piecewise differen- tiable on any interval J with I

J.

(c) The tent function B 1 (x) is piecewise differentiable on R because it IS linear on the intervals (-00, -1), (-1,0), (0,1), and (1,00).

(d) Any piecewise polynomial function is piecewise differentiable on R.

The following convergence result is due to Dirichlet. 1

Theorem 2.16. (Dirichlet) Suppose that f(x) has period a > 0 and is piece- wise differentiable on R. Then the sequence of partial sums of the Fourier series

IThe proof of Theorem 2.16 will not be given here but can be found for example in Walker, Fourjer Analysjs, Oxford University Press (1988), p. 19 (Theorem 4.5) and p. 48ff.

34 Chapter 2. Fourier Series

of f(x), {SN(X)}NEN, where

N S ( ) '" ( ) 21unx/a NX=L cne ,

n=-N

c(n) = ! j a f(t) e-27rmx/a dx, a 0

(2.8)

converges pointwise to the function f(x), where

f(x) =

[ lim f(t) + lim f(t) ] . 2 t-+x+ t-+x-

0.8

06

0.6

04

0.4

02

02

-0.5 0 0 5 1 1 5 2

-2 -1.5 1 -05 0 05 1 1 5 2 -2 -1 5

06

- I I ..... : .

- I I

08

04

02

.2 -1 5 1 -0.5 0 05 1 1.5 2

FIGURE 2.3. Partial sums SN(X) of the Fourier series f(x) from Ex- ample 2.13(a). Top left: N = 10, top right: N = 20, bottorn: N = 60.

Note that J(a) = f(a) if f(x) is continuous at x = a and that J(a) is the average value of the left- and right-hand limits of f(x) at x = a when f(x) has a jump discontinuity. If we assume that f (x) has no discontinuities, then we can make a stronger statement as in the following Theorem. 2

Theorem 2.17. Suppose that f(x) has period a > 0 and is continuous and

2The proof of Theorem 2.17 can be found in Walker, Fourjer Analysjs, Theorem 4.4, p.59.

2.1. Trigonometric Series 35

35

-3 -2 .1 0

3 4 5 6

-3 -2 -1 0

3 4 5 6

3 4 5

FIGURE 2.4. Partial sums SN(X) of the Fourier series J(x) from Ex- ample 2.13(b). top left: N = 10, top right: N = 20, bottom: N = 60.

piecewise differentiable on R. Then the sequence oj partial sums S N (x) given by (2.8) converges to J(x) in L'= on R.

What if the function f (x) is continuous but not piecewise differentiable? What can be said about the convergence of the Fourier series of such a function? It is by no means obvious that such functions exist, but they do. The most famous example is due to Weierstrass, who constructed a function continuous on R but not differentiable at any point of R. This function is defined by f(x) = L:71EN 3- 71 cos(3T1x). The Weierstrass lVI-test can be used to show that this function is continuous, but the proof that it is nowhere differentiable is hard. 3 By the Weierstrass l\I-test, the Fourier series of the Weierstrass function converges uniformly on R. However, this is not the case for all periodic func- tions, continuous on R. The following theorem is due to DuBois-Reymond. 4

3 An example of a continuous, nowhere differentiable function similar to the Weier- strass function, together with a very readable proof, can be found in Korner, Fourier Analysis, Cambridge University Press (1988), Chapter II. 4Excellent expositions and proofs of this theorem can be found in Korner, Fourier Analysis, Chapter 18. and also in Walker, Fourier Analysis, Appendix A.

36 Chapter 2. Fourier Series

Theorem 2.18. (DuBois-Reyrnond) There exists a function f(:r) continuous on R and with peT'iod 27r s'uch that the Fourier senes of f(.7:) diverges at .1' = O. That is, limN--+= SN(O) does not exist where SN(X) is given by (2.8).

In fact. it is possible to find a continuous, period 27T function whose Fourier series diverges at every rational multiple of 27T. o5 Therefore, it is impossible to make the statement that the Fourier series of every continuous function converges pointwise to that function. The next theorem, Theorem 2.19, is due to Fejer and makes a general statement about the convergence of the Fourier series of a continuous func- tion. The idea behind Fejer's Theorem is the following. Instead of looking at the partial sums (2.8), consider the arithmetic means of those partial SUlllS; that is. consider the sequence

1 N-l aN(x) = N L Sk(X), k=()

(2.9)

It is often the case that when the convergence of a sequence fails due to oscillation in the terms of the sequence, the arithmetic means of the sequence will have better convergence behavior. Take the simple example of the sequence {a( n)} nEN, where a( 71) = (-l)n. Clearly lim n --+= a( 71) does not exist because the terms simply oscillate back and forth between 1 and -1. However. if we consider the sequence of arithmetic means, {a(rz) }nEN. gi ven by

1 N a(n) = N L a(n), k=l

(2.10)

then

a (n)

{ _?/ n

if n is even. if n is odd.

so that lim n --+= a(71) = 0 (Exercise 2.25). If the original sequence {a(n)}nEN already converges. taking the arith- metic means will not affect the convergence; that is, if limn--+::xJ a(n) = a. then also lim n --+= a(n) = a (Exercise 2.26).

Theorem 2.19. (Fejer's Theorem) Let f(:r) be a function with period a > 0 continuo'll,s on R. and define for each n E N the function O'N(X) by (2.9), where Sk(:r) is given by (2.8). Then O'N(X) converges uniformly to f(x) on R as N -+ 00.

0vValker. Fourier Analysis, Appendix A.

2.2. Approximate Identities 37

Exercises

Exercise 2.20. Prove Euler's formula: For every x E R, e ix = CoS( X) + i sin( x).

Exercise 2.21. Prove that for every real number a,

J 1 . e zax dx = -;- e zax .

a

Exercise 2.22. Prove Theorem 2.7.

Exercise 2.23. Prove each of the statements made in Remark 2.12.

Exercise 2.24. Prove each of the statements made in Example 2.13.

Exercise 2.25. Show that if a(n) = (-l)n, n E N, then a(n) =

a(k) = { 0

f n

s even, n L -l/n If n IS odd. k=l

Exercise 2.26. Show that if limn--t= a( n) = a, then limn--t= a( n) = a, where a( n) is given by 2.10.

2.2 Approximate Identities

The notion of an approximate identity or summability kernel is used ex- tensively in all branches of analysis. The idea is to make precise the notion of a "delta function" that is well known and widely used by physicists. engineers, and mathematicians. The delta function, b (x), has the property that for any continuous function j (x),

L j(t) b(t) dt = j(O)

(2.11)

or more generally,

L f (t) J (x - t) dt = f ( x) for every x E R. From some elementary considerations (the reader may fill in the details), any function b(t) satisfying (2.11) must satisfy, L b(t) dt = 1.

b(t) = 0, t:f: 0

and

38 Chapter 2. Fourier Series

It is impossible for any ordinary function to satisfy these conditions since the Riemann integral of a function, f (x), vanishing at every x#-O must be zero. This must be true even under more general definitions of the integral (such as the Lebesgue integral). Therefore, c5(t) is not an ordinary function. So the question remains: How are we to make sense of this concept? There are two ways to do this.

1. Extend the definition of function. This has been done by L. Schwartz who defined the notion of a distribution or generalized function. 6

2. Approximate the delta by ordinary functions in some sense. This more elementary approach has its natural completion in the theory of distributions alluded to above, but can be understood without any advanced concepts. The idea is to replace the single "function" c5 (t) by a collection of ordinary functions {KT (t)}T>O such that for every continuous function f (x),

lim r f(t) KT(t) dt = f(O) T-+O+ JR

and more generally,

lim ( f(t) KT(x - t) dt = f(x), T-+O+ JR

where the limit is interpreted in some sense and described in Sec- tion 1.2.

The purpose of this section is to explain the theory of approximate iden- tities.

2. 2.1 Motivation from Fourier Series In order to further motivate the notion of an approximate identity, let us consider how one might prove Theorems 2.16 and 2.19.

Definition 2.27. For each kEN, and a > 0, define the Dirichlet kernel Dk(X) by

Dk(X) =

k L

21nrnx/a e .

(2.12)

rn=-k

See Figure 2.5.

6Very good expositions of this theory can be found in Horvath, An introduction to dis- tributions, The American Mathematical Monthly, vol. 77 (1970) 227-240, and Benedetto, Harmonic Analysis and Applications, CRC Press (1997).

2.2. Approximate Identities 39

Theorem 2.28. For each kEN, and a > 0, the Dirichlet kernel, Dk(X), can

be writt en as

Dk(X) = sin(7r(k + 1)x/a) sin( 7rX / a) , and for any period a function f (x),

(2.13)

1 l a Sk(X) = - f(x - t) Ddt) dt. a a

(2.14)

Proof: Equation (2.13) is an exercise (Exercise 2.38) and requires only the formula for summing a geometric series. As for equation (2.14),

Sk(X)

k L c( n) e27rinxja n=-k

ntk G 1" f(t) e -hinl/a dt) e 2 ,",x/a

1 l a k - f(t) L e 27rin (x-t)ja a 0 n=-k 1 l a - f ( t) D k (x - t) dt. a 0

The result follows by making the change of variables t r--t X - t in the above integral and remembering that both Dk(X) and f(x) have period a. D

Definition 2.29. For each n E N, and a > 0, define the Fejer kernel Fn (x) by

n-l Fn(x) =

L Ddx). k=O

(2.15)

See Figure 2.6.

Theorem 2.30. For each n E N, and a > 0, the Fejer kernel, Fn(x), can be

written as

Fn ( x) = ! ( Si

( 7rnx / a ) ) 2, n sm(7rx/a) and for any period a function f(x),

(2.16)

1 l a (J' ( n ) ( x) = - f (x - t) F n ( t) dt. a a

(2.17)

Proof: Equation (2.16) is an exercise (Exercise 2.39) and requires only the formula for summing a geometric series and some manipulation.

40 Chapter 2. Fourier Series

10

--- --- /" / \

....... / \ ./ -....... 1'-1 I --- 1 1 1 1 1 1 1 1 1

20

20

15

15

-5 -5 -05 -04 -03 -0.2 -0.1 0 01 02 03 04 05 -05 -0.4 -0.3 -02 -0.1 0 01 02 03 0.4 0.5

20

15

10

FIGURE 2.5. The Dirichlet kernel Dk(X) (2.14) for a = 1.

Equation (2.17) is also an exercise (Exercise 2.40), and the derivation is similar to (2.14). D From Theorems 2.28 and 2.30, we see that the proofs of Theorems 2.16 (Dirichlet) and 2.19 (Fejer) amount to showing that

1 l a - lim - f(x - t) Dk(t) dt = f(x) k--HXJ a 0

pointwise for every period a function f (x), piecewise differentiable on R, and

1 l a lim - f(x - t) Fn(t) dt = f(x) n-+= a 0

in L= on R for every period a function f(x) continuous on R. Such con- vergence results depend on properties of the sequences {Dk(X)}kEN and { Fn (x) }nEN. Consideration of the required properties of these sequences leads to the notion of an approximate identity or summability kernel.

2.2.2 Definition and Examples

Definition 2.31. A collection of functions {KT(x)}T>O on an interval I = ( -a, a) (a = 00 is permitted) is an approximate identity or a summability kernel

2.2. Approximate Identities 41

18 18 16 16 14 14 12 12 10 10

. - /1'\. V \ .J i\ T 1

-- 11\ II \ I \ I \

I \,

i .- T

o 0 -05 -0.4 -03 -0.2 -0.1 0 0 1 02 03 04 0 5 -05 -04 -0.3 -02 -0 1 0 0 1 0.2 0.3 04 05 20 18 16 14 12 10

i _I\J IA

2 o -05 -0.4 -03 -0.2 -0 1 0 0 1 02 03 0.4 0.5

FIGURE 2.6. The Fejer kernel Fn(x) (2.14) for a = 1.

on I if the following conditions hold. (a) For all T > 0, i: KT(X) dx = 1. (b) There exists M > 0 such that for all T > 0, 1: IKT(x)1 dx ::; M.

(c) For every 0 < <5 < a,

lim r IKT(x)1 dx = O. T-tO+ J8<Ixl<a

Example 2.32. (a) For 0 < T < 1, let KT(X) = 1/(2T)X C - T ,T)' Then {K T (X)}O<T<l is an approximate identity on (-1,1). (b) For 0 < T < 1, let KT(X) = (l/T)(l-lxl/T)X c - T ,T)(x), Then {K T (X)}O<T<l is an approximate identity on (-1,1). (c) More generally, let g(x) be any function LIon R and supported in (-a, a) (a = CXJ is permitted) such that L g(x)dx = 1.

42 Chapter 2. Fourier Series

For 0 < T < 1, let KT(X) = (l/T)g(x/T). Then {KT(X)}O<T<l is an ap- 2 proximate identity on (-a, a). If g(x) = e- 7fX , then the corresponding approximate identity {KT(X)}O<T<l is an approximate identity on R re- ferred to as the Gauss kernel.

(d) For T > 0, let ( sin( 7fX / (Ta)) ) 2 KT(X) = T . ( /) XC-a,a)(X)' SIn 7fX a

Then {KT(x)}T>O is an approximate identity on (-a, a). In fact, Kl/n(X) = Fn(x) XC-a,a)(x), where Fn(x) is the Fejer kernel defined by (2.16). (e) For T > 0, let

sin(7ft/(Ta)) KT(X) = . ( /) XC-a,a)(x), sm 7ft a

Then Kl/dx) = Dk(X) XC-a,a)(x), where Dk(X) is the Dirichlet kernel de- fined by (2.13). However, {KT(x)}T>O is not an approximate identity on (-a,a). (f) For each T > 0, let aT = [J

l (1 - t 2 )1/T dt] -1, and let KT(X) = 2 aT (1 - x2)1/T X[-l,l] (x).

Then {K T (x)} T >0 is an approximate identity on (-1, 1). Define Ln (x) = K1/n(x). Then the sequence {Ln(X)}nEN is referred to as the Landau ker- nel.

2.2.3 Convergence Theorems The idea behind approximate identities is to investigate the validity of the statement

lim r f(t) KT(x - t) dt = lim fT(X) = f(x). T-+O+ J R T-+O

(2.18)

Proving (2.18) will allow us to approximate arbitrary functions f(x) by functions of the form f T (x) that often can be made to have desirable prop- erties (see Corollary 2.37). In this subsection, we will consider mostly ap- proximate identities on R as those will be most useful to us later in the book. Let us first consider pointwise approximation.

Theorem 2.33. Let f(x) be L= on R and continuous at the point x = a. Suppose that {KT (x )}T>O is an approximate identity on R. Then

lim r f(t) KT(a - t) dt = f(a). T-*O+ J R

(2.19)

2.2. Approximate Identities 43

Proof: Note first that by a change of variables,

1 f(t) KT(a - t) dt = 1 f(a - t) KT(t) dt.

By Definition 2.31 (a),

f(a) = f(a) 1 KT(t) dt = 1 f(a) KT(t) dt.

Therefore, for any number b > 0,

If( a) - JR f( a - t) KT(t) dtl Il f(a) KT(t) dt - 1 f(a - t) KT(t) dtl Il (J(a) - f(a - t)) KT(t) dtl < llf(a) - f(a - t)IIKT(t)1 dt ( If(a) - f(a - t)IIKT(t)1 dt J 1 t l<6 + ( If(a) - f(a - t)IIKT(t)1 dt. (2.20) J 1 tl'26 Let E > O. Since f (x) is continuous at x = a, there is a b > 0 such that if It I < b, then If(a) - f(a - t)1 < E/(2M), where !vI is the upper bound in Definition 2.31(b). Hence for such a delta,

( If(a) - f(a - t)IIKT(t)1 dt J 1 tl<6

< E ( IKT(t)1 dt J 1 tl<6 ::; E lIKT(t)1 dt < (E/2ft1) M

E/2.

Since f(x) is Loo on R, If(y)1 ::; Ilflloo for all y E R. Therefore,

If(a) - f(a - t)1 ::; If(a)1 + If(a - t)1 ::; 211f1100,

for all t E R. Hence, for any b > 0,

( If(a) - f(a - t)IIKT(t)1 dt::; 211fll00 { IKT(t)1 dt

0 J 1 tl'26 J 1 tl'2 6

44 Chapter 2. Fourier Series

as T

0+ by Definition 2.31 (c). Therefore, there is a TO > 0 such that if o < T < TO, then

r If(a) - f(a - t)IIKT(t)1 dt < E/2. J 1 tl'26

Hence for 0 < T < TO,

If ( a) - .L f (a - t) K T ( t) dt I < E

and (2.19) follows.

D

Note that if KT(X) = (l/T) g(X/T) for some function g(x), LIon R and compactly supported, then the assumption that f(x) is L= on R is unnecessary. It would only be necessary to assume that f (x) is bounded on some small interval containing x = a, which follows from the continuity of f(x) at x = a (Exercise 2.42). Now let us consider uniform convergence.

Theorem 2.34. Let f(x) be L oo and uniformly continuous on R. Suppose that {K T (x )}7>0 is an approximate identity on R. Then

lim r f ( t) K 7 (x - t) dt = f ( x ) 7-*0+ J R

(2.21 )

in L oo on R.

Proof: Making the same estimate as in (2.20), it is sufficient to show that given E > 0, there is a b > 0 and an TO > 0 such that if 0 < T < TO, then

r If(x) - f(x - t)IIKT(t)1 dt + r If(x) - f(x - t)IIKT(t)1 dt < E. J 1 tl<6 J 1 tl'26 (2.22) As for the first term in (2.22), the uniform continuity of f(x) implies that there is a b > 0 such that if It I < b, then for all x E R, If(x) - f(x - t)1 < E/(2M), where M is the bound in Definition 2.31(b). Hence, as in the proof of Theorem 2.33,

r If(x) - f(x - t)IIKT(t)1 dt < (E/2M) M = E/2 J 1 tl<6

for all T > O. As for the second term in (2.22), since f(x) is L= on R,

r If(x) - f(x - t) IIKT(t) I dt ::; 211fll= r IKT(t) I dt, J 1 tl'26 J 1 tl'26

2.2. Approximate Identities 45

which converges to 0 as T ---t 0+ for any b > O. Hence there is a TO > 0 such that if 0 < T < TO, then r If(x) - f(x - t)IIKT(t)1 dt < E/2 J 1 tl'2 8 for all x E R, and (2.21) follows. D The condition that f(x) is uniformly continuous on R is satisfied if for example f(x) is c2 on R. More generally, it is satisfied if f(x) is CO on R and limlxl--+CXJ f(x) = 0 (Exercise 2.43). Lemma 2.35 (which we state without proof) establishes a very important property of functions referred to as the continuity of translation for £1 and £2 functions. This means that the translate of an £1 or £2 function remains very close to the original function provided that the translation is small enough. Here closeness is measured in the sense of the £1 or £2 norm. Continuity of translation is used to prove £1 and £2 convergence using approximate identities.

Lemma 2.35. (Continuity of Translation) Suppose that f(x) is piecewise con- tinuous on R. Then the following hold. (a) If f(x) is LIon R, then lim r If ( x) - f (x - t) I dx = O. t-'tO J R

(2.23)

(b) If f(x) is L 2 on R, then lim r If ( x) - f (x - t) 1 2 dx = O. t-'toJ R

(2.24)

As for £1 or £2 convergence in (2.18), the following theorem (which we state without proof) holds.

Theorem 2.36. (a) Suppose that f(x) is LIon R, and that {KT(x)}T>O is an approximate identity on R. Then

lim r f(t) KT(x - t) dt = f(x) T-'tO+ J R

(2.25)

in LIon R.

(b) Suppose that f(x) is L 2 on R, and that {KT(x)}T>O is an approximate identity on R. Then

lim r f(t) KT(x - t) dt = f(x) T-'tO+ JR

(2.26)

in L 2 on R.

46 Chapter 2. Fourier Series

12

12

,

C -::::::

:::.. J i un -- I I I I

08

08

06

06

04

04

0.2

02

o I I -02 0 02 0.4 0.6 0.8 1 1 2 -02 0 0.2 04 06 08 1 1 2

12

I i i I - I I I I I I I I

08

06

04

02

-02 0 02 04 06 08 1 1 2

FIGURE 2.7. Illustration of Lemma 2.35. Top Left: Graph of function f(x). Top Right: Graph of f(x) and f(x - t) where t = 1/16. Bottom: Graph of If(x) - f(x - t)l. Area of shaded region is small for small t.

The following approximation theorem is a consequence of Theorem 2.36.

Corollary 2.37. (a) Let f(x) be LIon R, and let E > O. Then there is a function g(x), C

on R such that

Ilf - glh < E.

(b) Let f(x) be L 2 on R, and let E > O. Then there is a function g(x), C

on R such that

Ilf - gl12 < E.

Proof: (a) By Theorem 1.5, there is a compactly supported function h(x), LIon R, such that Ilf - hill < E/2. Now, let KT(X) = (l/T)(l -lxl/T) XC-TJ)(x), T > 0 (see Example 2.32(b)). Then {KT(x)}T>O is an approximate identity on R. By Theorem 2.36(a), lim hT(x) = lim r h(t) KT(x - t) dt = h(x) T--+O+ T--+O+ JR

2.3. Generalized Fourier Series 47

in LIon R. Hence there is a TO > 0 such that Ilh TO - hilI < E/2. Let g(x) = hTo(x). That g(x) is compactly supported follows from Exercise 3.25, and that g(x) is CO on R follows from Theorem 3.18. The proof of (b) is similar (Exercise 2.44). D

Exercises

Exercise 2.38. Prove equation (2.13). (Hint: Recall that for any number I ,\"""N

I In = l-rN ) , L..,n=O I-r .

Exercise 2.39. Prove equation (2.16).

Exercise 2.40. Prove equation (2.17).

Exercise 2.41. Prove each of the statements made in Example 2.32.

Exercise 2.42. Prove that if J(x) is continuous at x = a, then there is a 5> 0 and a number M > 0 such that IJ(x)1 :::; M for all x E [a - 5, a + 5].

Exercise 2.43. (a) Prove that if J(x) is C2 on R, then J(x) is uniformly continuous on R. (b) Prove that if J(x) is Co on Rand limlxl--+= J(x) = 0, then J(x) is uniformly continuous on R.

Exercise 2.44. Prove Corollary 2.37(b).

2.3 Generalized Fourier Series

2.3.1 Orthogonality

Definition 2.45. A collection offunctions {gn(X)}nEN, L 2 on an interval I is a (general) orthogonal system on I provided that

(a) 1 gn(X) grn(X) dx = 0 ifn -I- m, and (b) 1 gn(X) gn(X) dx = 1Ign(X),2 dx > O.

Part (b) says in particular that none of the gn (x) can be identically zero. The collection {gn(X)}nEN is a (general) orthonormal system on I provided that it is an orthogonal system on I and

48 Chapter 2. Fourier Series

(b') 1 gn(X) gn(X) dx = 1Ign(X),2 dx = 1.

It is not necessary that the set {gn (x)} be indexed by N, and in fact we have seen an example (the trigonometric system) that is indexed by Z. In all future examples, the index set will either be specified or will be clear from the context. Whenever a generic system of functions is considered, the index set will be assumed to be N.

Remark 2.46. (a) Any orthogonal system can be normalized so that it becomes an orthonormal system. That is, if {gn (x)} is an orthogonal system, then we may define the functions

gn(x) = gn(x)/llgnI12'

Then the system {gn (x)} is an orthonormal system. (b) The Cauchy-Schwarz inequality guarantees that each of the integrals in Definition 2.45 exists as a finite number. That is, since f(x) and g(x) are L 2 on I,

11 f(x) g(x) dxl <; (jlf(x)1 2 dx t 2 (jlg(x)1 2 dx f2 < 00.

( c) Throughout the book, we will use inner product notation to represent the integrals in Definition 2.45. That is, we write for any functions f(x), and g(x) L 2 on I,

\f, g) = 1 f(x) g(x) dx.

This means in particular that

(I, J) = 1 f(x) f(x) dx = 1If(x),2 dx = Ilfll

.

Example 2.47. (a) Given any a > 0, the collection

{I, sin(nnxla), cos(nnx/a)}nEN

is an orthogonal system over [-a, a]. It is also orthogonal over [0,2a] and in fact over any interval I of length 2a. The collection

{ II . 1 } ;c:;-: , r;: sm (nnx I a), r;: cos ( nnx / a) y 2a yay a nEN

is an orthonormal system over [-a, a]. It is also orthonormal over [0,2a] and in fact over any interval I of length 2a.

2.3. Generalized Fourier Series 49

(b) Given any a > 0, the collections

{sin( 7rnx / a) }nEN

and

{cos( 7rnx / a) }nEN

are each orthogonal systems over [-a, a]. The collections

{

s in(7rnx/a) } va nEN

and

{

c os(7rnx/a) } va nEN

are each orthonormal systems over [-a, a]. (c) Given a > 0, the collection

{e27Tinx/a}nEZ

is an orthogonal system over [0, a], and in fact over any interval I of length a. The collection

{

e27Tinx/a } va nEZ is an orthonormal system over [0, a], and in fact over any interval I oflength a.

2.3.2 Generalized Fourier Series

Definition 2.48. Given a Junction J(x), L 2 on an interval I, and an or- thonormal system {gn(X)} on I, the (generalized) Fourier coefficients, {c(n)} oj J(x) with respect to {gn(X)} are defined by

c(n) = /, J(x) g(x) dx = (1,gn).

The (generalized) Fourier series oj J (x) with respect to {gn (x)} is

J(x) rv L (1, gn) gn(x). nEN

The fundamental problem is to determine under what circumstances the "rv" in the above definition becomes a "=" and, if so, in what sense the infinite series on the right side of the equality converges. It turns out that the most convenient form of convergence in this case is L 2 convergence on I.

Theorem 2.49. (Bessel's inequality) Let J(x) be L 2 on an interval I, and let {gn (x)} be an orthonormal system on I. Then

L 1(1,gn)1 2

/'IJ(x)1 2 dx = IIJII

. nEN I

(2.27)

50 Chapter 2. Fourier Series

The proof of Bessel's inequality will require the following lemma.

Lemma 2.50. Let {gn(x)} be an orthonormal system on an interval I. Then for every f(x), L 2 on I, and every N E N,

Ilf - t u , gn) gnli:

I1fll

- t IU,gn)I'

(2.28)

Proof: The proof is just a calculation making use of the orthonormality of {gn(x)}, j,lf(X) - t (J, g,,) g,,(X)1 2 d.r N N j, (f(X) -

(J,gn) g,,(X)) (f(X) -

(J,gn) g,,(X)) dx N j,lf(x) 12 dX-

(J,g,,) j, f(x) 9,,(X)dX N - L (f, gn) j, f(x) gn(x) dx n=l I N N +

y, g,,) (J, gm) j, g,,(x) gm(x) dx N j, If(x) 1 2 dx -

(J, g,,) (J, g,,) N N - L (f, gn) (f, gn) + L (f, gn) (f, gn) n=l n=l N N j, If(x)12 dx - 2

I(J, g,,)1 2 +

I(J, g,,)1 2 N j, If(x)12 dx -

1(J,gn)1 2 ,

which is (2.28).

D

Proof of Theorem 2.49: Let f(x) be given, and let {gn(x)} be an or- thonormal system. Then by Lemma 2.50, for each fixed N E N,

I N ' 2 N 0:; j, f(x)-

(J,g,,)g,,(x) dx= j,lf(x) 12 dX-

1(J,gnW

2.3. Generalized Fourier Series 51

Therefore, for all N E N,

N L I(f, g17)1 2

/'If(x)1 2 dx. 17=1 I

Since l(f,g17)12 2:: 0 for all n, the partial sums of the series L17EN l(f,g17)12 form an increasing sequence bounded above by II If(x)12 dx. Thus the series L17EN I (f, g17) 1 2 converges so that we can allow N to go to infinity. Thus,

L l(f,g17)12

/'If(x)!2 d X , 17EN I

which is (2.27).

D

Closely related to Lemma 2.50 is another very important inequality that will be very useful in the next subsection.

Lemma 2.51. Let {gn(X)} be an orthonormal system on I. Then for every f(x), L 2 on I, and every finite sequence of numbers {a(n)};;=l'

Ik - t a(n) gn Ii:

Ilf - t U ,9 n ) gnli: +

la(n) - U,9n)l'

Proof: Let f(x) be given, and let {g17(x)} be an orthonormal system. Then

Ilf -

a(n) gnll: /, (f(X) -

a( n) gn(X)) (f(X) - t a(n) gn(X)) d.T N /'If(x)1 2 dx -

a(n) /, f(x) gn(x) dx N N N -

a(n) /'f(x) gn(X) dX+

,

, a(n) a(rn) j,gn(x) gm(:r) dx N N N /'If(x) 12 dX -

a(n) (f,gn) -

a(n) (f,gn) +

a(n) a(n) N j, If(xW dx +

(la(n )1 2 - a(n) (f, gn) - a(n) (f, gn) + 1 (f, gnW)

N - L I(f, g17)1 2 17=1

52 Chapter 2. Fourier Series

N N /, If(x)1 2 dx - L I(f, gn)1 2 + L la(n) - (f, gn)1 2 I n=1 n=1 IIJ -

(J'9") 9nll: +

la(n) - (J,9,,)I'

by Lemma 2.50.

D

2.3.3 Completeness

We are now in a position to answer the fundamental question about Fourier series, namely: When is an arbitrary function equal to its Fourier series and in what sense does that Fourier series converge? The answer lies in the notion of a complete orthonormal system.

Definition 2.52. Given a collection of functions {gn(X)}, L 2 on an interval I, the span of {gn (x)}, denoted span {gn (x)}, is the collection of all finite linear combinations of the elements of {gn(X)}. In other words, f(x) E span{gn(x)} if and only if f(x) = L

=l a(n)gn(x) for some finite sequence {a(n)}

=l' Note that N is always finite but may be arbitrarily large.

Example 2.53. (a) Let PI denote the set of all polynomials on the in- terval I. Then PI = span{xn}

=o' (b) span {e-27Tinx }nEZ is the set of all period 1 trigonometric polynomials. (c) Let cp(x) = (1-lx[) X[

1,1] (x). Then span{ cp(x - n)}nEZ is the set of all functions that are (i) continuous on R, (ii) linear on intervals of the form [n, n + 1), nEZ, and (iii) compactly supported.

Remark 2.54. (a) For any collection of functions {gn(x)}, span{gn(x)} is a linear space; that is, it is closed under the formation of linear combi- nations. Specifically, if {fn(X)};;=l

span{gn(x)}, then for any finite se- quence {a(n)};;=1' the function f(x) = L:=1 a(n) fn(x) is in span{gn(x)} (Exercise 2.61). (b) The definition of span involves only finite sums. Without additional assumptions on the collection {gn(x)}, there is no guarantee that any sum of the form LnEN a( n) gn (x) will converge in any sense. For example, if gn (x) = x n - 1 for n EN, then the series L

=o n! x n does not converge except at x = 0, and the series L

=o 2- n x n does not converge if IxJ 2': 2. See also Theorem 2.55 below. ( c) Related to the notion of span {gn (x)} is the notion of the mean-square (or £2) closure of span{gn(x)}, denoted span {gn(x)} which is defined as follows. A function f (x) E span {gn (x)} if for every E > 0, there is a function g(x) E span{gn(x)} such that Ilf - gl12 < E.

2.3. Generalized Fourier Series 53

As a partial answer to the question of when finite sums can be replaced by infinite sums, we have the following theorem.

Theorem 2.55. Let {gn(X)} be an orthonormal system on an interval I. Then a function f(x), L 2 on I, is in span {gn (x)} if and only if

f(x) = L (1, gn) gn(X), nEN

(2.29)

in L 2 on I.

Proof: ({==) (2.29) is equivalent to the statement that

=IIJ -

(J, gn) gn II: = o.

Therefore, given E > 0, there is an N > 0 such that

IIJ -

(J,gn) g,1 < E

and f(x) E span {gn(x)}. (===}) Suppose that f(x) E span {gn(x)}, and let E > O. Then by definition there is a finite sequence {a(n)}

l' some No E N, such that

IIJ -

a(n) gnl1 2 < E.

By Lemma 2.51,

IIJ-

(J,gn)gnll: < IIJ-

(J,gn)gnll: +

la(n)-(J,gnW IIJ -

a(n) gn II:

<

2 E .

Since

{IIJ -

(J, gn) gn IIJ NEN

is a decreasing sequence (Exercise 2.63), it follows that for every N 2': No,

IIJ -

(J,gn) g,1 < E

54 Chapter 2. Fourier Series

and (2.29) follows.

D

If every function L 2 on I has a representation like (2.29), then we say that the collection {gn (x)} is complete on I. This means that every function L 2 on I is equal to its Fourier series in L 2 on I.

Definition 2.56. Let {gn (x)} be an orthonormal system on I. Then {gn(X)} is complete on I provided that every function f (x), L 2 on I, is in span {gn (x) }. A complete orthonormal system is called an orthonormal basis.

The next theorem gives several equivalent criteria for an orthonormal system to be complete.

Theorem 2.57. Let {gn(X)} be an orthonormal system on I. Then the fol- lowing are equivalent.

(a) {gn(x)} is complete on I. (b) For every f(x), L 2 on I,

f(x) = L (1, gn) gn(X) nEN

in L 2 on I.

(c) Every function f(x), C

on I, is in span {gn(x)}. (d) For every function f(x), C

on I,

Ilfll

= /, If(x)J2 dx = L 1(1. gn)1 2 . . I nEN