text:
Campus Clubs, Organizations Helped Seniors Find a True Community
Campus Clubs, Organizations He…
How did seniors find a second family on campus? By getting involved	Twitter
Editor's note: photo taken in 2018.
By Shannon Sigafoos
When arriving at college, students can find themselves looking for a new sense of purpose and community. Often, they find this by getting involved with causes that are personal to them. Though involvement can prove time consuming, it also provides a break from the rigors of coursework, establishes new friendships and connections, may help alleviate stress, and make the college experience more balanced and enjoyable. Here, seniors tell us how their clubs and organizations helped them feel a sense of belonging and adjust more easily to life at Lafayette:
Tanmaya Goenka '21 | Student Government
"Undertaking the role of Director of the Student Services committee within Student Government has been the crux of my learning curve since it has made me understand the importance of the community at Lafayette. I was nervous when I stepped into the position, but everyone, from my committee members to the executive board of Student Government, made me feel so comfortable and helped me adjust to this role in every way possible. Over two years, I formed a close-knit community within this diverse organization and I had the confidence that each of my committee members as well as the rest of Student Government had my back, and vice versa. Whether it was coping with my physical health or mental health, my peers were always there to support me. I am so grateful I was able to absorb such diverse perspectives and it made me confident about my voice and the impact each of us can have within the community–a lesson I will carry forward all my life!"
Zev Granowitz '21 | Hillel Society
"Hillel provided a home-away-from-home for me as a first year student at Lafayette. Ever since my first day on campus, I have known that Hillel is a place I could go to for support and to spend time with some of my best friends. I have already visited some of the Lafayette Hillel alumni and look forward to staying in touch with everyone after graduation."
Alisha Gangadharan '21 | Landis Center, Alpha Phi Omega
"Community engagement has definitely defined my experience at Lafayette. It's helped me discover the values and passions I'm most committed to, and all the people who I can share them with. Being part of the Landis and Alpha Phi Omega communities has given me the opportunity to get to know and grow alongside the most genuine, and straight up coolest people I know—some of whom I have the privilege to call my best friends. I'm forever grateful for that."
Mary Zimmerman '21 | Swim Team
"My swim team has certainly provided me with a family environment. The entire team has been a support-system for me since the first day that I arrived at Lafayette. The compassion, empathy, and willingness to understand and accept differences of each member has made me feel that I could be myself. I have made life-long friends that I know will be there for me at any time that I need them."
Melissa Niles '21 | Nia: Women of Purpose
"NIA has always represented a warm and secure space for me to authentically be myself. From freshman year to now, NIA has been a place where I realized that my voice deserves to be heard. At NIA meetings I could laugh, cry, sing, and network. In going to NIA events I realized that without my community I am nothing. It is at NIA that I met and bonded with some of my closest friends. My Lafayette experience would not be the same without my NIA sistas and siblings and I am so grateful that Lafayette introduced me to the true meaning of community."
Dominic Zhang '21 | Lafayette Ambassadors
"Being a part of the Ambassadors of Lafayette program in the past four years has really helped me to develop my interpersonal skills and introduced me to all kinds of cool people on campus that I otherwise would not have possibly met. I got involved as a tour guide my second semester on campus, which definitely helped me to meet the majority of my good friends today, and those friendships will go long on after graduation given all the memorable moments I have shared with my fellow tour guides, like giving tours on our annual XLC Day (admitted students day) here. I love Lafayette and see it as my second home, and in the end I'm just glad to have had that opportunity to present the best version of Lafayette to other prospective students who are seriously considering coming to Lafayette, like I did, four short years ago."
Brooke Paccoine '21 | landis Center
"Being a part of the Landis Center for Community Engagement is where I found my place at Lafayette. The work I have done has allowed me to give something back to a community that has given so much to me. It has connected me with other student leaders and staff who share the same passion for the work we do and have become lifelong friends because of it."
Zachary Whitney '21 | Lafayette Ambassadors
"I have absolutely loved working as an ambassador with admissions because it gave me an opportunity to talk every week about how much I adore this place that has become home to me. The other students involved in this program are all just as passionate about the school—and moreover the people that make up the community, and I'm certain they will continue to be in my life as we pursue the world beyond Lafayette. Not to mention, of course, the spectacular experience of meeting, recruiting, interviewing, and just altogether getting to know each newest class of leopards. Even if I won't be able to meet them on campus next year, I know a fantastic group of Pards will be joining us, and that's what makes being an Ambassador so rewarding."
Basit Baloghun '21 | Dyer Center
"The Dyer Center has been very critical in helping me explore my entrepreneurship mindset while allowing me to understand the importance and power of technology in business. My participation in a 3-day startup program and consequently the business pitch competition help me better understand how to iterate from an idea into a full-fledged business model. I learned about the importance of customer/market research in any idea, and I have been able to extend this knowledge to other areas of my life. The Dyer Center also funded my team trip to a Hackaton (HackNYU) where we were able to utilize our computer science background to create and innovate outside the 4 walls of a classroom."
Alyssa Megson '21 | Mosaic
"Participating in community service during my first year expanded my worldview. I became a MOSAIC Program Coordinator to become more involved and had the opportunity to meet Lafayette students that I would not have known otherwise. All the teams I worked with were composed of inspirational leaders. This led me to apply for a senior leadership position and become an Assistant Director in hopes that I could inspire others as I had been inspired. From my community engagement experiences at Lafayette, I will continue to cultivate my expanded worldview and leadership abilities in my professional and personal lives. I know that I will always have support from the Landis staff, students, and alumni."
Rachel Denham '21 | lafayette ambassadors
"As a senior, I look back to freshman year and remember how excited I was to be at Lafayette and share my love for Lafayette with others. One of the best ways to show that was through the Lafayette Ambassadors program, which I was lucky enough to join. With my peers in the program, I felt a collective passion for why we chose Lafayette and how we wanted to help prospective students find that as well. I have learned so much not only about Lafayette but also about why we are so passionate about Lafayette and where that comes from. Just last week I sat on the steps of Markle Hall with one of our Admissions Directors, Becca Pichetto, talking about my four years here and what comes next. It felt like a culmination of all the connections and memories I have made summed up into one hour. I'm grateful to say that I feel extremely thankful to be at Lafayette and to have formed the community that I was so excited about creating for myself."
Hoff Awards
Standout students, employees, programs, and student organizations honored
Support Systems
Seniors reflect on friendships formed at Lafayette
2021 Celebrations
Lafayette honors the talented, resilient Class of 2021
Special events and features celebrate this year's seniors
Categorized in: Class of 2021, Commencement 2021, Featured News, News and Features, Student Profiles, Students
Greg MacDonald says:
To all our graduating ambassadors, thank you for your service! We appreciate all that you have done to help us build our community.


"Ye have need of patience [cheerful endurance], that after ye have done the will of God, ye might receive the [fulfilment of] the promise. — Heb. 10:36.
THE Apostle is addressing the Christian Church—You who have left the world, who have accepted Christ as your Standard-bearer, as well as your Redeemer, and who are seeking to walk in His steps, and have made consecration of your life to the Lord — "Ye have need of patience." To a certain extent you did the will of God when you made your consecration to be dead with Christ. But that will of God was more deeply impressed upon you when you began to realize more than at first what this sacrifice would mean, and that only those who suffer with Christ shall reign with Him.
"After you were illuminated" you saw the matter clearly, and "endured a great fight of afflictions." This was well. But St. Paul goes on to show in the context that some, after having demonstrated their zeal for a certain time, become cold. They become weary in well doing. And he tells us that these thus cut themselves off from the favors, privileges and blessings belonging to the Church of Christ. His exhortation is that those who are still loyal to God at heart continue so and exercise patience, remembering that this is one of the cardinal graces of Christian character. Many have naturally a little love, a little gentleness, a little patience, a little meekness, etc. But after we begin to grow in the graces of the Holy Spirit, we need patience to control the flesh, the spirit of the world, the spirit of selfishness.
The will of God is in one sense of the word the standard of God—full perfection—that we should be like our Father which is in Heaven. But God remembers that we are fallen creatures, and that we cannot do perfectly. Our text does not mean that we must do the will of God in the perfect and complete sense; but rather, as the Apostle elsewhere says, ours is a reasonable service. When we present our bodies living sacrifices to God, it is our reasonable service. God does not expect us to do that which is impossible.
What is this will of God? Stated in concrete form, "This is the will of God [concerning you], even your sanctification." (I Thess. 4:3.) As the Apostle teaches, consecration is a full and complete setting apart. If we do such a setting of ourselves apart at the beginning, then the Father sanctifies us—begets us as New Creatures, and sets us apart. So we have, first of all, our setting of ourselves apart; and then God's acceptance by our begetting of the Holy Spirit as New Creatures, and His continued work in us.
We are doing the will of God when we fully consecrate ourselves to Him, and attain a place in the New Creation. But He wills to put us to the test. How much do we love God? How sincere are we? A soldier in an army might be loyal in time of quiet, but how would he be in time of stress? Would he desert the flag then, or would he prove himself a good soldier? He would need a great deal of patience. If he says he loves his country, his endurance and faithfulness will be tested in her time of need. He must go on picket duty; he must sometimes do menial work. He must endure wearisome marches, and many privations. All these things are required of a faithful soldier. If he is faithful, he is likely to be promoted, honored, for his faithful service.
So we are tested as to our loyalty. What are we willing to endure for Christ's sake? How fully are we submitted? How deep does our submission go? Are we wholly in harmony with the will of the Lord? Is our interest merely superficial, or does it enter fully into our hearts? The question is not merely, Shall we make the consecration?—but after the Christian has taken all of these preliminary steps, to what extent will he manifest patient endurance and obedience and loyalty?
God puts us to these tests because He has great honors to bestow on those who will be overcomers. They are to be a select company, and these will receive the Promise. As the Apostle says, it is after we have proved our loyalty to the very last, that we shall receive the Promise; i.e., its fulfilment.
When, where, what is the Promise? Undoubtedly the promise will be received in the resurrection. The promise includes all that God has in reservation for them that love Him—that love Him more than they love houses and lands, or children, or parents, or friends, or husbands or wives, or self, or any other thing.
The particular promise that the Apostle refers to here is The Promise. All our hopes and blessings are centered in the original Promise made to Abraham, when God brought him out of the land of Chaldea into the land of Canaan. God promised Abraham that in His Seed should all the families of the earth be blessed. That has been the great Promise for encouragement to the Seed, to give them patience and fortitude. This is the essence of the Promise—that those who receive the Promise shall be the Seed of Abraham to bless the world. The faithful in Christ will be associated with Him in His Kingdom—will have the honor of blessing all the families of the earth under this Kingdom. Every creature of God shall then be brought to a knowledge of His Truth, and shall have the opportunity of being restored, if he will, to perfection, to all that was redeemed on Calvary.
Now the opportunity is different. Now the selection is being made of those who will inherit the Promise as the Seed of Abraham. "If ye be Christ's, then are ye Abraham's Seed, and heirs according to the Promise." The Apostle is in our text urging that we continue to be Christ's and to abide in Him. All those who thus remain in Him to the end will be glorified with Him. In order to remain faithful, we must have His spirit of devotion.
St. James exhorts the Church saying, "Take, my brethren, the Prophets, who have spoken in the name of the Lord, for an example of suffering affliction, and of patience." Those whom the Apostle addressed already knew of the sufferings of Jesus. They already knew of the faithfulness of the Apostles. And now he was calling the attention of their minds to something additional. He is urging, Look back into the past, and see that patient endurance has been characteristic of all who have lived holy lives. These examples should be lessons of encouragement to us, in addition to those we have in the living brethren around us!
Then there is always something to be gained in casting the mind backward. The things close at hand are too near to be seen in their proper light. It was fitting that the Apostle should call attention to those faithful ones of the past, so that we might be encouraged to note what God desires. In those who are His, He desires a willingness to endure patiently and loyally, thus manifesting true character, that which greatly pleases Him.
As we look back over the Old Testament record of the Prophets, we notice that many of them displayed this very quality referred to by the Apostle as loyalty to the Lord, a willingness to suffer afflictions for His sake, and not as experiences brought through chance upon them by the people. We see Moses—how willing he was to suffer affliction because of his faith in the Promise made to Abraham and his conviction that the Promise would come true. He preferred to suffer with the people of God rather than to live at ease in the royal family of Pharaoh, into which he had been adopted.
We see in Job another example of patient endurance of tribulation and of strong opposition for a considerable time. We see the same in Jeremiah—how much his faithfulness cost him of hardship, and how patient he was. We see the same in Daniel the Prophet—his faithfulness to the Lord, his patient endurance of whatever God permitted to come against him. And so with others of the Prophets. And we read that their experiences were written for our admonition, our instruction. Although they belong to one Dispensation and we to another, yet their experiences furnish us good lessons.—I Cor. 10:6, 11.
Applying these lessons to ourselves, we may say that to whatever extent we may be privileged to speak the Word of God and to suffer persecution therefore, if we take it with patience, it will bring us a corresponding blessing and commendation from the Lord. But we cannot think it would be pleasing to Him if, when we suffer, we think, Oh, how terrible, terrible, terrible! Such an attitude would not be taking His Word for it, that "All that will live godly in Christ Jesus shall suffer persecution," and that all things shall work out for our good.—2 Tim. 3:12; Rom. 8:28.
When Job was rich, prosperous, God tested him by taking from him all his family, all his wealth, his health, and even allowing his wife to turn against him. Yet in all this Job did not turn against God. He did indeed express wonder, but he looked to the Lord in faith and said, "Though worms shall destroy this body, yet in my flesh shall I see God." I shall yet receive the manifestation of His favor, and learn what He means by these experiences, these afflictions, coming upon me. "Though He slay me, yet will I trust in Him."—Job 13:15.
After his testings had been accomplished, God gave him back children, houses, lands, friends. And these coming in abundance shadowed forth the blessings of Restitution—how the tribulations of mankind will eventually work out for good to those who will love God. If those who are now suffering affliction because of their loyalty to the Lord, because of their trust in His arrangements, will take afflictions and trials joyfully, these will surely work out good to them—"a far more exceeding and eternal weight of glory."


GANGNEUNG, South Korea - You could exhaust a dictionary's supply of superlatives to describe what happened in the Olympic pairs short program Wednesday, and it would be a thoroughly justified verbal outpouring.
From the utterly breathtaking brilliance of winners Wenjing Sui and Cong Han of China to the exuberance with which North Koreans Tae Ok Ryom and Ju Sik Kim threw off all the political and historical weight on their shoulders for the performance of a lifetime, there never has been so much relentless excellence at a pairs competition in a global championship.
"The level of skating was a pure joy to watch," said two-time Olympic singles champion Katarina Witt, a commentator for German TV. "The quality of the lifts and throws and jumps was so high.
"It was sport at a high level and entertainment at a high level, particularly with the variety of music used -- from Ed Sheeran to the Beatles to Tchaikovsky."


What is the Optimal Time Management?
In our modern world, time management is one of the most important skills. No matter how successful we are, or how much we get done, it can feel like we're always falling behind. This learning program doesn't just tell you about great time management skills: it teaches them to you through fun activities. At the end of this activity that anyone can do anywhere at any time of the year, team members will be able to juggle even the busiest work schedule through the optimization of their time management.
No matter the size of your group, we'll break it into small ones. Then, we'll go through several time management tasks. These will all be in the guise of a business simulation, so the real world application will be obvious and apparent.
All the facets of time management, how to prioritize one task over again, delegating the right task to the right person, being able to tackle multiple tasks at once, will each be explored and tested.
Showcase! Groups will present to the others what they worked on, how it went and more.
"Time management is one of the hardest things for everyone at our company, from management on down. The interactive games were fun and they kept us on our toes, too!"


text:
New York Times Subscription Deals
5 savers
Up to 73% off The New York Times Basic Digital Subscription
Verified Used 49 Times Today
Details: Save up to 73% on your Basic Digital Access subscription to The New York Times, which provides you access to NYTimes.com and all NYT apps, unlimited article access, and discounts to select sites.
Save even more with The Wall Street Journal
50% off 1-Year Subscription to WSJ All Access Digital
Verified 6 hours ago Used 45 Times Today
New York Times Subscriptions starting at $1/Week for 1 Year
Verified on 1/09/20 Used 11 Times Today
Keep up with what's happening in the world with respected news coverage by The New York Times.
Free and unlimited access to NYTimes.com, plus NYTimes smartphone and tablet apps
Get access to the NYT Crossword, NYT Cooking, bonus subscriptions, and more with select plans
Home delivery prices may vary based on your address and delivery frequency
Introductory offer is billed as $4 every 4 weeks for one year
Free First Month with The New York Times Crosswords Subscription
Details: Get access to daily Crossword puzzles, over 20 years of archived crosswords, and daily Mini Puzzles. One month Free, then just $6.95 per month! Available for both Android and iOS devices.
Popular The New York Times Subscriptions
Up to 73% Off Up to 73% off The New York Times Basic Digital Subscription ---
Sale New York Times Subscriptions starting at $1/Week for 1 Year ---
Sale Free First Month with The New York Times Crosswords Subscription ---
Start Shopping at NYTimesAtHome.com
$1 for 15 Weeks with Wall Street Journal Student Discount
From The Economist
$12 for 12 Weeks of The Economist Digital Subscription
Monday to Friday Print Delivery & USA TODAY's e-Newspaper...
From The Boston Globe
50% off your Home Delivery Subscription
From Investor's Business Daily
Get a 4 Week Free Trial of IBD Digitral + Weekly Print Combo
About The New York Times
Get all the news that's fit to print from The New York Times, one of America's premier newspapers.
Students pay $1/wk.
Take advantage of The New York Times student discount that's valid as long as you're a student.
Get digital access for multiple users if you're part of a school, hotel, library or corporation.
Today's The New York Times Offers
Free Shipping Offers 0
The New York Times Company, a leading, elite media company, includes The New York Times, the International Herald Tribune, The Boston Globe, 15 other daily newspapers, WQXR-FM, and more than 50 Web sites, including NYTimes.com, Boston.com and About.com. And with New York Times coupons, you can access these top-notch news sources for less.
Get quality daily news from The New York Times. Since its first circulation in 1851, the paper has won 106 Pulitzer Prizes – more than any news organization – and NYTimes.com boasts over 30 million viewers per month. In each issue of The New York Times, you'll find:
The top news stories in NYC, the United States, and the world
Opinion pieces from world-renowned editorialists
Business, Arts, Science, Style, and Home features
If you don't have time to grab your morning paper or if you're on the go, you can also access your New York Times subscription online and on your mobile devices like the iPhone and the Android.
Q: How do I get NY Times digital access?
A: All print subscribers automatically get free digital or online access to the NY Times. What's more, you'll also get free access to the NYTimes.com smartphone and tablet apps for your iPhone, iPad, Nook, Android, and other devices.
Q: What is the NY Times digital subscription for students?
A: Anyone who is currently a college student, professor, faculty, or administrator at an accredited university can take advantage of special NY Times pricing. With a NY Times student subscription, you can pay as little as $1 per week for as long as you're a student . You'll need to use your .edu email address to start a digital account.
Q: Is NY Times online access free?
A: Yes and no. If you're not a NY Times subscriber, you'll only be able to access certain content for free – currently, 20 free articles per month and access to the home page and blogs. Anything beyond that, you'll need to subscribe to see. The good thing is, with a digital or home delivery subscription, you get unlimited access to all of the NY Times digital content.
Q: What are the benefits of a NY Times subscription?
A: Get your news when you want it, how you want it. With NY Times home delivery, you get the convenience of having the news delivered to your door. But in addition, you'll also get access to the NYTimes.com website, as well as mobile access on your smartphone or tablet. In addition, digital subscribers will get the benefits of online, phone, and iPad or tablet access. The NY Times is one of the most trusted sources of news in the world, and its worldwide news coverage is a can't-miss.
When you're looking for a dependable newspaper that includes all the day's news that's 'fit to print,' you won't find a more elite and pristine source than The New York Times. Save money on a The New York Times subscription with The NY Times subscription deals.
Comments for The New York Times (16)
OffersChrisR commented on 10/14/19
@campbellnellie Please contact the merchant directly from their site regarding your query. Offers.com is a third party advertiser only, not the merchant.
campbellnellie commented on 10/14/19
Can delivery be made overseas
@beverlyaldridge Thank you for your comment. Offers.com is a third party advertiser only, not the merchant. Once you click through an offer you are directed to the merchant's website to complete your transaction with them. Please contact the merchant directly regarding your query. We apologize for any confusion.
beverlyaldridge commented on 12/9/17
What is the Special-Price for The NYTimes SUNDAY- ONLY, DELIVERED?
DonTConme commented on 10/23/17
Through the NYT Android App an annual Basic Digital Subscription is $129.00. This is far cheaper than what is offered here.
stevensjeanette commented on 7/22/17
What are your rates for a Friday, Saturday and Sunday only subscription?
OffersChrisR commented on 4/10/17
@sboone Offers.com is an advertising medium for our merchant partners, not the merchant. We have emailed you with additional information about contacting this merchant directly, as they are the ones you will need to speak with regarding your issue. We apologize for any confusion.
sboone commented on 4/9/17
3rd party: pls forward: Current subscriber considering cancelation: spoke w/nytcustomer service rep yesterday: no satisfaction; requested supervisor: denied. And I only had a question about a fee on my bill. What do subscribers do who hv serious issues?????? S Boone, Tac
@swtybillpark We post the offers as we receive them, and once you click through an offer link you are taken to the merchant's site to complete your transaction with them. Currently we do not show any here on our page for the New York Times referencing the $2 Sunday delivery you mentioned. If you have a question about subscriptions, please contact the merchant directly, as Offers.com is a third party advertiser only. Thank you, and have a great day!
swtybillpark commented on 9/27/16
Where is the $2 sunday delivery in your offer?
@TCHOYT2003 You will need to contact the merchant directly regarding your query, as Offers.com is a third party advertiser only and not the merchant. We do not have any role in their delivery of items, subscriptions or services. Once you click an offer you are directed to the merchant site to complete your transaction with them. We apologize for any confusion. Thank you for choosing Offers.com!
TCHOYT2003 commented on 9/23/16
If I subscribe on what day of the week will I receive the paper and how will it be delivered?
OffersSamantha commented on 7/8/15
@aribot0808 If you follow the link below you can see they are offering a 12 week subscription for just 99 cents. http://www.nytimes.com/subscriptions/Multiproduct/lp88U46.html?campaignId=4FWFJ&%3B__KEYWORDS__=%24%7BkeywordText%7D&%3B__CAMP__=4FWFJ For any further questions please contact The New York Times directly. Thank you!
aribot0808 commented on 7/8/15
I received an offer for 75% off for 12 weeks in the mail. I don't see this offer any where. Is it still available? Thank You! Alfredo Ribot ([email protected])
DaOffersGuy commented on 8/29/14
NYTimes currently has a special offer for $2 per week for the first month, with full access to their website from your computer or through their smartphone or tablet apps. Check out nytimes.com/twodollars for full details.
austinshopper commented on 8/29/14
Hi, I saw this coupon my friend got in the mail for The NYT. Has anyone seen it?
Get New The New York Times Offers

redpajama_set_name: RedPajamaCommonCrawl


At The White Lighthouse we think a white hall table is an essential piece of furniture in your hallway which is why we offer many different options to buy... as a place to display white flowers, photos, and a lamp to add decor and interest to you hallway.
When you enter your home, a place to set your keys and the post that has arrived that day.
The size of the table greatly depends on how much room you have in your hall as you do not want an oversized piece of furniture that you have to squeeze past or are always knocking into with schoolbags or as you bring your laundry upstairs. If you have a small hall, a lamp table is probably the best option.
Preferably you will want a hall table with some drawers for the storage of pens, notepaper, keys letters and coins. Baskets under the table are perfect to for children to throw their shoes into as they come into the house instead of leaving on the floor!
A very simple but elegant pure white table for hall or living room interiors. The New England Range is a crisp matt white range of furniture for a classic look to any room. As perfectly suitable for coastal interiors as it is for Victorian or country homes.
The Colonial White half moon hallway table with a shelf, a perfect stylish and affordable option for smaller hallway and entry way spaces.
A white hall table will lighthen your hallway, does not matter what colour your walls are, a white table will alway complement any colour... white on white for a coastal New England look, or a dark wall with a white table for a clubhouse feel. For any home style, coastal, New England, country, cottage, French, suburban, a white hall table will match in perfectly.
Add some artificial white Hydrangea to great you and your guests on arrival, and they will last for years. A very cost effective way to have flowers all year round.
By adding a coat stand in your hall you will not need much more furniture to complete.
A simple tray table is enough if you just want to add a lamp to your hall.
A chest of drawers is a great idea for a hallway console table asit gives massive amounts of the extra storage needed.
New England White Chest of Drawers for greater amounts of storage.


The written word came to life at the International Writer's Festival in Perth on Saturday. Under sunny skies, perched along the historic Tay Basin, the Crystal Palace was home to nearly a dozen authors showcasing their work and chatting with locals and visitors alike. The event was part of a weekend of activities which included a writing contest and some notable guest speakers from the writing community.
Many thanks to the Chamber of Commerce along with the Ottawa International Writers Festival, the Farmer's Market, the Town of Perth, the Perth & District Union Public Library, the BIA , Heritage Perth, The Book Nook and many local businesses who organized and sponsored this first-time event in the newly formed Perth Chapter of the well-known International Writers Festival.
We were pleased to be situated with author Gene Bassett to our right and with Irene Spence and Ron Shaw to our left. Irene was the recipient of the Queen's Diamond Jubilee award for her many years of service in promoting history and heritage and is active at Archives Lanark as well as the Lanark County Genealogical Society.
It was a pleasure meeting author and instructor Lee Ann Eckhardt Smith, member of the Board of Directors of the Ottawa International Writers Festival. Lee Ann, along with Leslie Wallack of The Book Nook presented each author with a smart black carry-all featuring the Writer's Festival logo, a writing journal, pen and decorative ceramic keepsake.
One of the highlights of the day were visits from Carol-Ann McDougall and husband Ken who are building their dream home along the picturesque shores of Rideau Ferry. Another delightful visitor was an old friend and neighbour from the third line – Dr. Steve Scott. It was a wonderful surprise to connect once again and have a quick chat.
The day was a great success and we look forward to future events with the International Writer's Festival!
This entry was posted in Author Events & Book Signings, Uncategorized and tagged Arlene Stafford-Wilson, August 23 2014, Canadian Author, Heritage Perth, Lanark County author, Ottawa International Writer's Festival, Perth author, Perth Business Association, Perth Chapter International Writers Festival, Perth Tourism, The Book Nook Perth Ontario, Writers Festival.
← International Writer's Festival Comes to Perth!
It was a good day for local authors though I was disappointed more were not allowed to participate. Perhaps next year the organizers will post for interest. Congratulations with your display though.
Thank-you! The sponsors and organizers were very pleased with the turnout for the first ever Writer's Festival in Perth, and hopefully next year they will expand on the number of authors invited to participate.


text:
La plaza Mayor de Zamora es un espacio de planta rectangular ubicado en la ciudad de Zamora (España). Este espacio se ve ocupado en la actualidad parcialmente en su lado oeste por la iglesia de San Juan de Puerta Nueva. La plaza cambia de nombre en numerosas ocasiones durante el siglo XIX.

Hasta mediados del siglo XX la plaza se encontraba delimitada por las casas adosadas a dicha iglesia, y no fue hasta los años cincuenta, que debido a la restauración de la iglesia, se derribaron los edificios que tenía adosados, produciéndose el ensanche de la plaza al espacio que se conoce en la actualidad. Anteriormente el espacio que había tras la iglesia se denominaba "plazuela de San Miguel", tras el ensanche quedó unificado en el nombre genérico de plaza Mayor gracias a los diseños del arquitecto español José María Aparicio. En el año 2004 se celebra el quinto centenario de la plaza.

En el suelo enlosado de la plaza se puede distinguir un trazado paralelo a la iglesia de San Juan de Puerta Nueva formado por losas que están colocadas en diferente posición a las demás. Estas fueron colocadas para indicar donde estuvo ubicado el  primer recinto amurallado de la ciudad, incluyendo la forma de una semicircunferencia para reseñar donde se encontraba uno de sus cubos defensivos de la muralla zamorana.

Historia

En la antigüedad aparecían espacios abiertos en torno a las puertas principales de la muralla zamorana. En estos espacios extramuros se organizaban mercados y ferias. La iglesia de San Juan de Puerta Nueva es construida antes de que existiera la plaza como espacio. Las Leyes de Toledo promulgadas de los Reyes Católicos hicieron que se edificaran edificios consistoriales en cada ciudad castellana. A mediados del siglo XVII se construyen soportales en la fachada del edificio. En el año 1484 se comienza la edificación del Ayuntamiento Viejo, el primitivo Concejo promulgó el comienzo de las obras de la plaza por nivelar los taludes y expropiar casas. Ya en el año 1766 se edifica en el lado este la Casa de las Panaderas mientras que en sus soportales se establecen espacios para la venta del pan. A comienzos del siglo XX se construye frente al viejo ayuntamiento el edificio que será el nuevo ayuntamiento. Tras la iglesia de San Juan de Puerta Nueva se encuentra un espacio que se denomina "Plazuela de San Miguel".

A lo largo del siglo XIX la plaza cambia de nombre en diversas ocasiones, por ejemplo en el periodo 1812-1814 se denomina plaza de la Constitución, en 1814-1820 se nombra como plaza del Rey, en 1820-1823 se menciona como plaza de la Constitución, en 1823-1836 como plaza del Rey y en 1836 como plaza de Isabel II. En 1879 se produce un gran incendio en la plaza. Se construye un nuevo edificio que será conocido como el Nuevo Ayuntamiento, en el que se incluyeron espacios de cafetería y juzgados. En la iglesia de San Juan con los años se han ido construyendo adosados diferentes viviendas. Junto al lado oriental de la plaza se construyen en 1912 unas viviendas para Juan Gato, diseñadas por el arquitecto catalán Francisco Ferriol. El arquitecto José María Aparicio propone una nueva ordenación de la plaza. Este plan unifica la denominada "plazuela de San Miguel", tras la iglesia de San Juan, en un único espacio. El derribo de las casas adosadas a la iglesia se produce en los años setenta.

Características

La plaza ocupa un espacio de planta cuasi-rectangular en el que desembocan nueve calles. Las calles importantes de la ciudad como la calle Balborraz, la calle de Santa Clara finalizan esta plaza, así como la calle de los Herreros. En lado meridional se encuentra el edificio del Ayuntamiento viejo que se construye en tiempos del reinado de los Reyes Católicos.

Véase también
 Iglesia de San Juan de Puerta Nueva - Iglesia del siglo XII ubicada en un lateral del espacio de la plaza.

Referencias

Enlaces externos

Zamora
Plazas de Zamora

redpajama_set_name: RedPajamaWikipedia


Proximity: point distance on a one-to-one basis?
I have two sets of points: starting and ending locations. I want to find the distance between each starting point and its endpoint (i.e., no repeats: A to B, C to D, E to F, etc.). Is there a function for this?
If your data are in a table then take a look at the XY to Line tool.


text:
Men more likely to 'make do and mend'
The study suggests people under the age of 35 are the most likely to take up sewing.
Published: 16:55 Monday 18 January 2016
Men are more likely to mend their clothes than women, according to new research.
And the study suggests people under the age of 35 are the most likely to take up sewing - with Kirstie Allsopp, Fearne Cotton and David Beckham the top three celebrities most likely to set them stitching.
The survey of 1,000 adults found that 50 per cent more men than women say they would mend an item of clothing.
The research, for retailer Hobbycraft, showed that only 40 per cent of women would mend an item, compared to 60 per cent of men.
The survey also revealed that those aged 18 to 35 are the most likely to fix an item of clothing.
While sewing has seen a revival in popularity in recent years, thanks to TV programmes such as The Great British Sewing Bee, the survey showed that one in 10 people are still unable to do basic tasks such as sewing on a button, sewing on a zip or turning up or mending a hem.
And more than half (55 per cent) admitted they would throw away items of clothing rather than try to mend them. The most wasteful areas are London (66 per cent), closely followed by the East Midlands (61 per cent) and the East of England (59 per cent).
But those in Northern Ireland are the most thrifty with 60 per cent of people revealing they wouldn't throw away any items of clothing they could fix, closely followed by those in the North East (59 per cent) and Wales (57 per cent).
While sewing and knitting are the most popular crafts, the hobbies people are most likely to take up in 2016 are drawing and painting, confirming the trend for adult colouring which saw sales increase by 991 per cent in 2015 and is set to be even bigger in 2016 with four in 10 saying they plan to take it up this year.
Katherine Paterson, customer development director at Hobbycraft, said: "It's brilliant to see men trying their hand at sewing and with Patrick Grant at the forefront of the sewing bee and an increase in male contestants over the years the gender stereotype are really been broken down.
"The popularity of sewing has been growing for a number of years now, so it's surprising how many people are still unable to do simple sewing tasks.
"We launched the sew simple campaign at Hobbycraft last year and the purpose of that is to help everyone learn the basics, from setting up and maintaining your first sewing machine, to taking the sewing machine driving test with our instore experts helping shoppers master basic stitches."
She added: "The trend for crafting sees no sign of abating and there are a few reasons for that.
Traffic officer was driving at almost 120mph while 'following' young motorist, inquest told
"Firstly, programmes like The Great British Bake Off brought crafting to the forefront of the nation's minds and the absolutely brilliant thing about programmes like this is that they inspire the professionals to keep going and they inspire the non-crafters to get started.
"We've also seen a desire for the younger generation to get crafty; this started to emerge as a way to make money, save money and as alternative to going out in a time when people were more conscious of their spending.
"The final and probably the most important factor in the growing popularity of craft is the accessibility. Crafting went from something you learnt from your parents, grandparents or teachers to something you could learn at the touch of a button.
"Blogs, online 'How To Guides' and YouTube has done wonders for crafting because it's made the tricks of the trade easier to master."
Here are 11 ways Wakefield has changed since the year 2000
Police cordon off Wakefield street as investigation into firearms incident continues
Five vehicle collision closed major Wakefield road
Paedophile filmed girlfriend carrying out sex act with dog and fantasised about having children so he could rape them
The lost pubs of Wakefield - how many of these do you remember?
More from Wakefield Express


text:
Q: How to modify font using MailApp Google App script I created a script that sends mail using MailApp.sendEmail() but I can't find how to change the font.
I am using this format:
var subject = xxxxxx;
var body = 'xxxxxx' + var + 'xxxxx' + var + 'xxxx';
var recipient = xxxxxx;
MailApp.sendEmail(recipient, subject, body);

How do I modify the body style?

A: You can do it with the advanced parameter htmlBody
Sample:
  var body='<p style = "font-family:georgia,garamond,serif;font-size:16px;font-style:italic;"> xxxxxx </p> ';
  var subject = 'xxxxxx';
  var recipient = 'xxxxxx';
  MailApp.sendEmail(recipient, subject,"",{htmlBody: body});

redpajama_set_name: RedPajamaStackExchange


What options do I need to do this? I'm using the knockout binding.
Right now, after I paste in a value, it selects the item but does not close the drop down.
I am attaching a sample based on your requirement.
while this does allow the igCombo to close after pasting in a value, now the drop down immediately closes after you try to open again because it has a value already and triggers the close.
I modified the event to check if the "e.originalEvent.type == "paste""
I am glad that you've managed to resolve your issue and I believe that other community members may benefit from this as well.
Please, let me know if I may be of further assistance.


If you are scheduling an upcoming surgical procedure, be sure that all of the healthcare professionals involved are aware of any prescription medications, over-the-counter (OTC) medications and even herbs or home remedies that you are currently taking. This is especially true if you are taking the prescription Fosamax® (alendronate) or another drug in the bisphosphonate family. If not stopped at least three months prior to surgery, you are at risk for developing osteonecrosis—literally bone death. For oral surgery patients, most cases reported occurred after a dental extraction; however, it can also occur spontaneously and may be associated with pain at the site.
Learn more in Fosamax® and Surgery.


text:
\subsection{A General Framework for Analyzing Random Initialization in First $(H-1)$ Layers}
In this section we provide a self-contained framework to analyze the Gram matrix at the initialization phase.
There are two main objectives.
First, we provide the expression of the Gram matrix as $m \rightarrow \infty$, i.e., the population Gram matrix.
Second, we quantitatively study how much over-parameterization is needed to ensure the Gram matrix generated by the random initialization.
The bound will depend on number of samples $n$ and properties of the activation function.
This analysis framework is fully general that it can explain fully connected neural network, ResNet, convolutional neural considered in this paper and other neural network architectures that satisfy the general setup defined below.

We begin with some notations.
Suppose that we have a sequence of real vector spaces
\begin{equation*}
	\mathbb{R}^{p^{(0)}}\rightarrow \mathbb{R}^{p^{(1)}}\rightarrow \cdots\rightarrow \mathbb{R}^{p^{(H)}}.
\end{equation*}
\begin{rem}
For fully-connected neural network and ResNet, $p^{(0)} = p^{(1)} =\ldots =p^{(H)} = 1$.
For convolutional neural network, $p^{(h)}$ is the number of patches of the $h$-th layer.
\end{rem}

For each pair $(\mathbb{R}^{p^{(h-1)}},\mathbb{R}^{p^{(h)}})$, let $\mathcal{W}
\subset \mathcal{L}(\mathbb{R}^{p^{(h-1)}},\mathbb{R}^{p^{(h)}})= \mathbb{R}^{p^{(h)}\times p^{(h-1)}}$ be a linear subspace.
\begin{rem}
For convolutional neural network, the dimension of $\mathcal{W}$ is the filter size.
\end{rem}
In this section, by Gaussian distribution $\mathcal{P}$ over a $q$-dimensional subspace $\mathcal{W}$, we mean that for a basis $\{\vect{e}_1,\ldots,\vect{e}_q\}$ of $\mathcal{W}$ and $(v_1,\ldots,v_q) \sim N(\vect{0},\mathbf{I})$ such that $\sum_{i=1}^{q}v_i\vect{e}_i \sim \mathcal{P}$.
In this section, we equip one Gaussian distribution $\mathcal{P}^{(h)}$ with each linear subspace $\mathcal{W}^{(h)}$.
By an abuse of notation, we also use $\mathcal{W}$ to denote a transformation. For $\mathbf{K} \in \mathbb{R}^{p^{(h-1)}\times p^{(h-1)}}$, we let \[
\mathcal{W}^{(h)}\left(\mathbf{K}\right) = \mathbb{E}_{\mathbf{W}\sim\mathcal{P}^{(h)}}\left[
\mathbf{W}\mathbf{K}\mathbf{W}^\top
\right].
\]

We also consider a deterministic linear mapping $\mathcal{D}^{(h)}:\mathbb{R}^{n^{(h-1)}}\rightarrow \mathbb{R}^{n^{(h)}}$.
For this section, we denote $\mathcal{D}^{(1)} = \mathbf{0}$, i.e., the zero mapping.
\begin{rem}
	For full-connected neural networks, we take $\mathcal{D}^{(h)}$ to be the zero mapping.
	For ResNet and convolutional ResNet, we take $\mathcal{D}^{(h)}$ to be the identity mapping.
\end{rem}
Let $\rho^{(1)},\cdots,\rho^{(H)}$ be a sequence of  activation functions over $\mathbb{R}$.
Note here we use $\rho$ instead of $\sigma$ to denote the activation function because we will incorporate the scaling in $\rho$ for the ease of presentation and the full generality.

Now we recursively define the output of each layer in this setup.
In the following, we use $h \in [H]$ to index layers, $i \in [n]$ to index data points, $\alpha, \beta, \gamma \in [m]$ or $[d]$ to index channels (for CNN) or weight vectors (for fully connected neural networks or ResNet).
\begin{rem}
$d=1$ for fully connected neural network and ResNet and $d \ge 1$ for convolutional neural network because $d$ represents the number of input channels.
\end{rem}
We denote $\vect{X}^{(h),[\alpha]}_i$ an $p^{(h)}$-dimensional vector which is the output at $(h-1)$-th layer.
We have the following recursive formula
	\begin{align*}
	\vect{X}^{(1),(\alpha)}_i =&\rho^{(h)}\left(\sum_ \beta \mathbf{W}^{(h),(\alpha)}_{(\beta)}\vect{X}^{(h-1),(\beta)}_i\right)\\
	\vect{X}^{(h),(\alpha)}_i =&\mathcal{D}^{(h)}(\vect{X}^{(h-1),(\alpha)}_i)+\rho^{(h)}\left(\frac{\sum_ \beta \mathbf{W}^{(h),(\alpha)}_{(\beta)}\vect{X}^{(h-1),(\beta)}_i}{\sqrt{m}}\right)
	\end{align*}
	where $\mathbf{W}^{(h),(\alpha)}_{(\beta)}$  is $p^{(h)}\times p^{(h-1)}$ matrix generated according to the following rule
	\begin{itemize}
		\item for $h=1$, $\mathbf{W}^{(h),(\alpha)}_{[\beta]}$ is defined for $1\le \alpha\le m$ and $1\le \beta\le d$; for $h>1$, $\mathbf{W}^{(h),(\alpha)}_{(\beta)}$ is defined for $1\le \alpha\le m$ and $1\le \beta\le m$;
		\item the set of random variables $\{\mathbf{W}^{(h),(\alpha)}_{(\beta)}\}_{h,\alpha,\beta}$ are independently generated;
		\item for fixed $h, \alpha, \beta$, $\mathbf{W}^{(h),(\alpha)}_{(\beta)}\sim \mathcal{P}^{(h)}$.
	\end{itemize}
\begin{rem}
Choosing $\rho^{(h)}(z)$ to be $\relu{z}$ and $\mathcal{D}^{(h)}$ to be the zero mapping, we recover the fully-connected architecture.
Choosing $\rho^{(h)}(z)$ to be $\frac{c_{res}}{H}\relu{z}$ and $\mathcal{D}^{(h)}$ to be the identity mapping, we recover ResNet architecture.
\end{rem}
\begin{rem}
	Note here $\vect{X}_i^{(h)} = \vect{x}_i^{(h)}\sqrt{m}$ for $h \ge 1$ and
	$\vect{X}_i^{(h)} = \vect{x}_i^{(h)}$ for $h = 0$ in the main text.
	We change the scaling here to simplify the calculation of expectation and the covariance in this section.
\end{rem}

With these notations, we first define the population Gram matrices recursively.

\begin{defn}\label{defn:pop_kernel}
We fix $(i,j) \in [n] \times [n]$, for $h=1,\ldots,H$.
The population Gram matrices are defined according to the following formula
\begin{align}
\mathbf{K}^{(0)}_{ij}=&\sum_\gamma (\vect{X}^{(0),[\gamma]}_i)^\top \vect{X}^{(0),[\gamma]}_j, \nonumber  \\
\vect{b}_i^{(0)} =&\vect{0},\nonumber\\
\mathbf{K}^{(h)}_{ij}=&\mathcal{D}^{(h)}\mathbf{K}^{(h-1)}_{ij}\mathcal{D}^{(h)\top}+\mathbb{E}_{(\vect{U},\vect{V})}\left(\rho(\vect{U}) \mathcal{D}^{(h)}(\vect{b}^{(h-1)}_j)^\top+(\mathcal{D}^{(h)}(\vect{b}^{(h-1)}_i)) \rho(\vect{V})^\top+\rho(\vect{U})\rho(\vect{V})^\top)\right), \nonumber \\
\vect{b}_i^{(h)}=& \mathcal{D}^{(h)}(\vect{b}_i^{(h-1)})+\mathbb{E}_{\mathbf{U}}\rho^{(h)}(\mathbf{U}),\label{eqn:kernel_defn}
\end{align}
where
\begin{equation}
(\mathbf{U},\mathbf{V})\sim N\left(\vect{0},\left(\begin{array}{ccc}
\mathcal{W}\left(\mathbf{K}^{(h-1)}_{ii}\right)  & \mathcal{W}\left(\mathbf{K}^{(h-1)}_{ij}\right) \\
\mathcal{W}\left(\mathbf{K}^{(h-1)}_{ji}\right) & \mathcal{W}\left(\mathbf{K}^{(h-1)}_{jj}\right)\\
\end{array}\right)\right). \label{eqn:uv_dist}
\end{equation}
\end{defn}
Notice that the Gram matrix of the next layer $\mathbf{K}^{(h)}$ not only depends on the previous layer's Gram matrix $\mathbf{K}^{(h-1)}$ but also depends on a ``bias" term $\vect{b}^{(h-1)}$.

Given the population Gram matrices defined in Equation~\eqref{eqn:kernel_defn} and~\eqref{eqn:uv_dist}, we derive the following quantitative bounds which characterizes how much over-parameterization, i.e., how large $m$ is needed to ensure the randomly generated Gram matrices is close to the population Gram matrices.

\begin{thm}\label{thm:main_general_framework}
With probability $1- \delta$ over the $\left\{\mathbf{W}_{(\beta)}^{(h),(\alpha)}\right\}_{h,\alpha,\beta}$, for any $ 1\le h\le H-1,1\le i,j\le n,$
	\begin{equation}
	\label{eqn:K_bound}
	\norm{\frac{1}{m}\sum\limits_{\alpha=1}^m(\vect{X}^{(h),(\alpha)}_i)^\top \vect{X}^{(h),(\alpha)}_j-\mathbf{K}^{(h)}_{ij}}_\infty\le \mathcal{E}\sqrt{\frac{\log (Hn\max_h p^{(h)}/\delta)}{m}}
	\end{equation}
	and any $h\in[H-1], \forall 1\le i\le n,$
\begin{equation}
	\label{eqn:b_bound}
\norm{\frac{1}{m}\sum_{\alpha=1}^m \vect{X}^{(h),(\alpha)}_i-\vect{b}^{(h)}_i}_\infty\le \mathcal{E} \sqrt{\frac{\log (Hn\max_h p^{(h)}/\delta)}{m}}
\end{equation}
The error constant $\mathcal{E}$ satisfies there exists an absolute constant $C>0$ such that
\begin{align*}
\mathcal{E}\le C \left(\prod_{h=2}^{H-1} \left(A_{(h)}+\Lambda_{(h)}\mathfrak{W}+ C_{(h)}A_{(h)}B\mathfrak{W}+C_{(h)} A_{(h)}\sqrt{\mathfrak{W}_{(h)}M}\right)\right)\times
		 \max\{\mathfrak{W}\sqrt{(1+ C^2_{(1)})M^2},\sqrt{C^2_{(1)}M}\}
\end{align*}
	where $M,B,\Lambda_{(h)},C_{(h)},A_{(h)},\mathfrak{W}_{(h)}$ are defined by:
	\begin{itemize}
		\item $M=1+100\max_{i,j,p,q,h}|\mathcal{W}^{(h)}(\mathbf{K}^{(h-1)}_{ij})_{pq}|$,
		\item $A_{(h)}=1+\max\left\{\|\mathcal{D}^{(h)}\|_{L^\infty\rightarrow L^\infty},\|\mathcal{D}^{(h)}(\cdot)\mathcal{D}^{(h)\top}\|_{L^\infty\rightarrow L^\infty}\right\}$,
		\item $B=1+100\max_{i,p,h}|\vect{b}^{(h)}_{ip}|$,
		\item $C_{(h)}=\abs{\rho(0)}+\sup_{x\in \mathbb{R}}\abs{\rho'(x)}$,
		\item $\Lambda_{(h)}$ is a constant that only depends on $\rho^{(h)}$,
		\item $\mathfrak{W}_{(h)}=1+\|\mathcal{W}^{(h)}\|_{L^\infty\rightarrow L^\infty}$.
	\end{itemize}
\end{thm}
\begin{rem}
	\label{rem:constants}
For fully-connected neural networks, we have $M = O(1),A_{(h)} =0, B = O(1), C_{(h)} = O(1), \Lambda_{(h)} = O(1), \mathfrak{W}_{(h)} = O(1)$, so we need $m = \Omega\left(\frac{n^2\log(Hn/\delta)2^{O(H)}}{\lambda_0^2}\right)$.
For ResNet, we have $M = O(1),A_{(h)} =1, B = O(1), C_{(h)} = O(\frac{1}{H}), \Lambda_{(h)} = O(\frac{1}{H}), \mathfrak{W}_{(h)} = O(1)$, so we need $m = \Omega\left(\frac{n^2\log(Hn/\delta)}{\lambda_0^2}\right)$.
The convolutional ResNet has the same parameters as ResNet but because the Gram matrix is $np \times np$, so we need $m = \Omega\left(\frac{n^2p^2\log(Hnp/\delta)}{\lambda_0^2}\right)$.
\end{rem}

\begin{proof}[Proof of Theorem~\ref{thm:main_general_framework}]
The proof is by induction.
For the base case, $h=1$, recall
\begin{equation*}
\vect{X}^{(1),[\alpha]}_i=\rho^{(1)}(\sum_ \beta \mathbf{W}^{(1),(\alpha)}_{(\beta)}\vect{X}^{(0),(\beta)}_i).
\end{equation*}

We define
\begin{equation*}
\vect{U}^{(1),(\alpha)}_i=\sum_\beta W^{(1),(\alpha)}_{(\beta)}\vect{X}^{(0),(\beta)}_i.
\end{equation*}

By our generating process of $\left\{\mathbf{W}_{(\beta)}^{(h),(\alpha)}\right\}_{h,\alpha,\beta}$,
the collection $\{\vect{U}_i^{(1),(\beta)}\}_{1\le i\le n,1\le \beta\le m}$ is a mean-zero Gaussian variable with covariance matrix:
\begin{align*}
	&\mathbb{E} \vect{U}_i^{(1),(\alpha)} \left(\vect{U}_j^{(1),(\beta)}\right)^\top\\
	=&\mathbb{E}\sum_ {\gamma, \gamma'} \mathbf{W}_{(\gamma)}^{(1),(\alpha)} \vect{X}^{(0),(\gamma)}_i  \left(\vect{X}^{(0),(\gamma')\top}_i\right)^\top \left(\mathbf{W}_{(\gamma')}^{(1),(\beta)}\right)^\top\\
	=&\delta_{\alpha \beta}\mathcal{W}^{(1)}\left(\sum_\gamma \left(\vect{X}^{(0),(\gamma)}_i\vect{X}_j^{(0),(\gamma)}\right)^\top\right)\\
	=&\delta_{\alpha \beta}\mathcal{W}^{(1)}(\mathbf{K}^{(0)}_{ij})
\end{align*}
Therefore, we have
\begin{align*}
\mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m\vect{X}_i^{(1),(\alpha)}\vect{X}_j^{(1),(\alpha)\top}\right]=&\mathbf{K}^{(1)}_{ij}\\
\mathbb{E}\left[\frac{1}{m}\sum_{i=1}^m\vect{X}_i^{(1),(\alpha)}\right]=&\vect{b}^{(1)}_i.
\end{align*}
Now we have calculated the expectation.
Note since inside the expectation is an average, we can apply standard  standard Bernstein bounds and Hoeffding bound and obtain the following concentration inequalities.
With probability at least $1- \frac{\delta}{H}$, we have
\begin{align*}
\max_{i,j}\norm{\frac{1}{m}\sum_{i=1}^m\vect{X}_i^{(1),(\alpha)}\vect{X}_j^{(1),(\alpha)\top}-\mathbf{K}^{(1)}_{ij}}_\infty\le&\sqrt{\frac{16(1+2 C^2_{(1)}/\sqrt{\pi})M^2\log(4Hn^2(p^{(1)})^2/\delta)}{m}},\\
\max_{i,p}\abs{\frac{1}{m}\sum_{\alpha=1}^m \vect{X}_{ip}^{(1),(\alpha)}- \vect{b}_{ip}^{(1)}}
\le&\sqrt{\frac{2C^2_{(1)}M\log(2np^{(1)}H/\delta)}{m}}
\end{align*}

Now we prove the induction step.
 Define for $1\le h\le H$
\begin{align*}
	\hat{\mathbf{K}}_{ij}^{(h)}=&\frac{1}{m}\sum_\gamma \vect{X}^{(h),(\gamma)}_i\left(\vect{X}^{(h),(\gamma)}_j\right)^\top\\
	\hat{\vect{b}}_{i}^{(h)}=&\frac{1}{m}\sum_\gamma \vect{X}^{(1),(\gamma)}_i
\end{align*}
In the following, by $\mathbb{E}^{(h)}$ we mean taking expectation conditioned on first $(h-1)$ layers.

Now suppose that Equation~\eqref{eqn:K_bound} and~\eqref{eqn:b_bound} hold for $1\le l\le h$ with probability at least $1- \frac{h}{H}\delta$, now we want to show the equations holds for $h+1$ with probability at least $1- \delta/H$ conditioned on previous layers satisfying Equation~\eqref{eqn:K_bound} and~\eqref{eqn:b_bound}.
Let $l=h+1$.
recall
\begin{align*}
\vect{X}^{(l),(\alpha)}_i=\mathcal{D}^{(l)}(\vect{X}^{(l-1)})+\rho^{(l)}\left(\frac{\sum_\beta \mathbf{W}^{(l),(\alpha)}_{(\beta)}\vect{X}^{(l-1),(\beta)}_i}{\sqrt{m}}\right).
\end{align*}
Similar to the base case, denote
\begin{align*}
\mathbf{U}^{(l),(\alpha)}_i=\frac{\sum_ \beta \mathbf{W}^{(l),(\alpha)}_{(\beta)}\vect{X}^{(l-1),(\beta)}_i}{\sqrt{m}}.
\end{align*}
Again note that $\{\vect{U}_i^{(l),(\beta)}\}_{1\le i\le n,1\le \beta\le m}$ is a collection of mean-zero Gaussian variables with covariance matrix:
\begin{align*}
\mathbb{E} \left[\vect{U}_i^{(1),(\alpha)} \left(\vect{U}_j^{(1),(\beta)}\right)^\top\right]=
\delta_{\alpha \beta}\mathcal{W}^{(l)}(\hat{\mathbf{K}}_{ij}^{(l-1)})
\end{align*}

Now we get the following formula for the expectation:
\begin{align*}
\mathbb{E}^{(l)}[\hat{\mathbf{K}}^{(l)}_{ij}]=&\mathcal{D}^{(l)}\hat{\mathbf{K}}^{(l-1)}_{ij}\left(\mathcal{D}^{(l)}\right)^\top+\mathbb{E}_{(\vect{U},\vect{V})}\left(\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_j)+(\mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})+\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V}))\right)\\
\mathbb{E}^{(l)}\hat{\vect{b}}_i^{(l)}=&\mathcal{D}^{(l)}(\hat{\vect{b}}_i^{(l-1)})+\mathbb{E}_{\vect{U}}\rho^{(l)}(\vect{U})
\end{align*}
with
\begin{align*}
		(\vect{U},\vect{V})\sim N\left(\vect{0},\left(\begin{array}{ccc}
		\mathcal{W}^{(l)}(\hat{\mathbf{K}}_{ii}^{(l-1)}) & \mathcal{W}^{(l)}( \hat{\mathbf{K}}_{ij}^{(l-1)})\\
		\mathcal{W}^{(l)}( \hat{\mathbf{K}}_{ji}^{(l-1)}) & \mathcal{W}^{(l)}( \hat{\mathbf{K}}_{jj}^{(l-1)})\\
		\end{array}\right)\right)
\end{align*}
Same as the base case, applying concentration inequalities, we have with probability at least $1- \delta/H$,
\begin{align*}
\max_{ij}\|\mathbb{E}^{(l)}\hat{\mathbf{K}}^{(l)}_{ij}- \hat{\mathbf{K}}^{(l)}_{ij}\|_\infty\le &\sqrt{\frac{16(1+2 C^2_{(l)}/\sqrt{\pi})M^2\log(4Hn^2(p^{(l)})^2/\delta)}{m}},\\
\max_{i}\|\mathbb{E}^{(l)}\hat{\vect{b}}^{(l)}_{i}-\hat{\vect{b}}^{(l)}_{i}\|_\infty\le &\sqrt{\frac{2C^2_{(l)}M\log(2np^{(1)}H/\delta)}{m}}
\end{align*}

Now it remains to bound the differences
\begin{align*}
\max_{ij}\norm{\mathbb{E}^{(l)}\hat{\mathbf{K}}_{ij}^{(l)}- \mathbf{K}_{ij}^{(l)}}_{\infty} \text{ and }
\max_{i}\norm{\mathbb{E}^{(l)}\hat{\vect{b}}^{(l)}_{ij}- \vect{b}^{(l)}_{ij}}_{\infty}
\end{align*}
which determine how the error propagates through layers.

We analyze the error directly.
\begin{align*}
&\norm{\mathbb{E}^{(l)}\hat{\mathbf K}_{ij}^{(l)}- \mathbf K_{ij}^{(l)}}_{\infty}\\
\le& \norm{\mathcal{D}^{(l)}\hat{\mathbf K}^{(l-1)}_{ij}\mathcal{D}^{(l)\top}-\mathcal{D}^{(l)}\mathbf{K}^{(l-1)}_{ij}\mathcal{D}^{(l)\top}}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf{A}}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_j)-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf{A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)}_{\infty}\\
&+\norm{\mathbb{E}_{(U,V)\sim \hat{\mathbf{A}}}(\mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf A}(\mathcal{D}^{(l)}(\vect{b}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf{A}}}\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V}))-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf{A}}\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V}))}_{\infty}\\
\le&  \norm{\mathcal{D}^{(l)}\hat{\mathbf K}^{(l-1)}_{ij}\mathcal{D}^{(l)\top}-\mathcal{D}^{(l)}\mathbf{K}^{(l-1)}_{ij}\mathcal{D}^{(l)\top}}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_j)-\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf{A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}(\vect{a}^{(l)}(\hat{\vect{b}}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})-\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf{A}}}(\mathcal{D}^{(l)}(\vect{b}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{A}}(\mathcal{D}^{(l)}(\vect{b}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf{A}}(\mathcal{D}^{(l)}(\vect{b}^{(l-1)}_i))^\top \rho^{(l)}(\vect{V})}_{\infty}\\
&+\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf{A}}}\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V}))-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf{A}}\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V}))}_{\infty}
\end{align*}
where we define
\begin{align*}
\hat{\mathbf A}=\left(\begin{array}{ccc}
\mathcal{W}^{(l)}(\hat{\mathbf
K}_{ii}^{(l-1)}) & \mathcal{W}^{(l)}( \hat{\mathbf K}_{ij}^{(l-1)})\\
\mathcal{W}^{(l)}( \hat{\mathbf K}_{ji}^{(l-1)}) & \mathcal{W}^{(l)}( \hat{\mathbf K}_{jj}^{(l-1)})\\
\end{array}\right)
\text{and }
\mathbf{A}=\left(\begin{array}{ccc}
\mathcal{W}^{(l)}(\mathbf
K_{ii}^{(l-1)}) & \mathcal{W}^{(l)}(\mathbf K_{ij}^{(l-1)})\\
\mathcal{W}^{(l)}( \mathbf K_{ji}^{(l-1)}) & \mathcal{W}^{(l)}( \mathbf K_{jj}^{(l-1)})\\
\end{array}\right)
\end{align*}

By definition, we have
\begin{align*}
\|\mathbf A-\hat{\mathbf A}\|_\infty\le &\mathfrak{W}\max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty}\text{ and }\\
\norm{\mathcal{D}^{(l)}\hat{\mathbf K}^{(l-1)}_{ij}\mathcal{D}^{(l)\top}-\mathcal{D}^{(l)}\mathbf K^{(l-1)}_{ij}\mathcal{D}^{(l)\top}}_{\infty}\le & A_{(l)}\max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty}.
\end{align*}
We can also estimate other terms
\begin{align*}
&\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\hat{\vect{b}}^{(l-1)}_j)-\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{ \mathbf A}}\rho^{(l)}(U)^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)}_{\infty}\\
\le &\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}\left(\hat{\vect{b}}^{(l-1)}_j-\vect{b}^{(l-1)}_j\right)}_{\infty}\\
\le &C_{(l)}A_{(l)}\sqrt{\mathfrak{W} \max_{ij}\|\hat{\mathbf K}_{ij}^{(l)}\|_\infty}\max_{i}\norm{\hat{\vect{b}}^{(l-1)}_{ij}- \vect{b}^{(l-1)}_{ij}}_{\infty}\\
\le &C_{(l)} A_{(l)}\sqrt{\mathfrak{W}_{(l)}M}\max_{i}\norm{\hat{\vect{b}}^{(l-1)}_{ij} - \vect{b}^{(l-1)}_{ij}}_{\infty},
\end{align*}

	\begin{align*}
&\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top \mathcal{D}^{(l)}(\vect b^{(l-1)}_j) - \mathbb{E}_{(\vect U,\vect V)\sim \mathbf A}\rho^{(l)}(\vect U)^\top \mathcal{D}^{(l)}(\vect{b}^{(l-1)}_j)}_{\infty}\\
\le &A_{(l)}B C_{(l)}\|\mathbf A-\hat{\mathbf A}\|_\infty \\
\le  &A_{(l)}BC_{(l)} \mathfrak{W} \max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty},
	\end{align*}
and
	\begin{align*}
&\norm{\mathbb{E}_{(\vect{U},\vect{V})\sim \hat{\mathbf A}}\rho^{(l)}(\vect{U})^\top\rho^{(l)}(\vect{V})-\mathbb{E}_{(\vect{U},\vect{V})\sim \mathbf A}\rho^{(l)}(U)^\top\rho^{(l)}(V)}_{\infty}\\
\le &\Lambda_{(l)}\|\mathbf A-\hat{\mathbf A}\|_{\infty}\\
\le &\Lambda_{(l)}\mathfrak{W}\max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty}.
	\end{align*}
where we have used Lemma~\ref{lem:2p2pperb}.

Putting these estimates together, we have

\begin{align*}
&\max_{ij}\|\mathbb{E}^{(l)}\hat{\mathbf K}_{ij}^{(l)}- \mathbf K_{ij}^{(l)}\|_\infty\\
\le& \left(A_{(l)}+\Lambda_{(l)}\mathfrak{W}+2C_{(l)}A_{(l)}B\mathfrak{W}\right)\max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty}+2C_{(l)} A_{(l)}\sqrt{\mathfrak{W}_{(l)}M}\max_{i}\|\hat{\vect b}^{(l-1)}_{ij}- \vect{b}^{(l-1)}_{ij}\|_\infty\\
\le& \left(A_{(l)}+\Lambda_{(l)}\mathfrak{W}+2C_{(l)}A_{(l)}B\mathfrak{W}+2C_{(l)} A_{(l)}\sqrt{\mathfrak{W}_{(l)}M}\right)\left(\max\limits_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_{\infty}\vee\max_{i}\|\hat{\vect b}^{(l-1)}_{ij}- \vect b^{(l-1)}_{ij}\|_\infty\right)
\end{align*}
	and
\begin{align*}
\max_{i}\norm{\mathbb{E}^{(l)}\hat{\vect b}^{(l)}_{ij}- \vect b^{(l)}_{ij}}_{\infty}&\le
\Lambda_{(l)}\mathfrak{W}\max_{ij}\norm{\hat{\mathbf K}^{(l-1)}_{ij} - \mathbf K^{(l-1)}_{ij}}_\infty+
A_{(l)}\max_{i}\norm{\hat{\vect b}^{(l-1)}_{ij}- \vect b^{(l-1)}_{ij}}_\infty\\
&\le (A_{(l)}+\Lambda_{(l)}\mathfrak{W})\left(\max_{ij}\|\hat{\mathbf K}^{(l-1)}_{ij}-\mathbf K^{(l-1)}_{ij}\|_\infty\vee\max_{i}\|\hat{\vect b}^{(l-1)}_{ij}- \vect{b}^{(l-1)}_{ij}\|_\infty\right).
\end{align*}

These two bounds imply the theorem.
\end{proof}

\subsection{From $\mathbf{K}^{(H-1)}$ to $\mathbf{K}^{(H)}$}
Recall $\mathbf{K}^{(H)}$ defined in Equation~\eqref{eqn:kernel_mlp},~\eqref{eqn:kernel_resnet} and~\eqref{eqn:kernel_conv_resnet}.
Note the definition of $\mathbf{K}^{(H)}$ is qualitatively different from that of  $\mathbf{K}^{(h)}$ for $h = 1,\ldots,H-1$ because $\mathbf{K}^{(H)}$ depends on $\mathbf{K}^{(H)}$ and $\sigma'(\cdot)$ instead of $\sigma(\cdot)$.
Therefore, we take special care of $\mathbf{K}^{(H)}$.
Further note $\mathbf{K}^{(H)}$ for our three architectures have the same form and only differ in scaling and dimension, so we will only prove the bound for the fully-connected architecture.
The generalization to ResNet and convolutional ResNet is straightforward.

\begin{lem}\label{lem:H_concen_mlp}
For $(i,j) \in [n] \times [n]$, define
\begin{align*}
\hat{\mathbf{K}}_{ij}^{(H-1)} = \hat{\mathbf{K}}_{ij}^{(H-1)}\mathbb{E}_{\vect{w}\sim N(\vect{0},\mathbf{I})}\left[\sigma'(\vect{w}^\top \vect{x}_i^{(H-1)}(0))\sigma'(\vect{w}^\top \vect{x}_j^{(H-1)}(0))\right].
\end{align*}
and suppose $\abs{\hat{\mathbf{K}}_{ij}^{(H-1)}-\mathbf{K}_{ij}^{(H-1)}} \le \frac{c\lambda_0}{n^2}$ for some small constant $c>0$.
Then if $m = \Omega\left(\frac{n^2\log(n/\delta)}{\lambda_0^2}\right)$, we have with probability at least $1-\delta$ over $\{\vect{w}_r^{(H)}(0)\}_{r=1}^m$ and $\{a_r(0)\}_{r=1}^m$, we have $\norm{\mathbf{G}^{(H)}(0)-\mathbf{K}^{(H)}}_{op} \le \frac{\lambda_0}{4}$.
\end{lem}
\begin{proof}[Proof of Lemma~\ref{lem:H_concen_mlp}]
We decompose
\begin{align*}
\mathbf{G}^{(H)}(0) - \mathbf{K}^{(H)}
=  \left(\mathbf{G}^{(H)}(0) - \hat{\mathbf{K}}^{(H)} \right)+ \left(\hat{\mathbf{K}}^{(H)} -\mathbf{K}^{(H)}\right).
\end{align*}
Recall $\mathbf{G}^{(H)}$ defined in Equation~\eqref{eqn:H_mlp}.
Based on its expression, it is straightforward to use concentration inequality to show if $m = \Omega\left(\frac{n^2\log(n/\delta)}{\lambda_0^2}\right)$, we have
\begin{align*}
\norm{\mathbf{G}^{(H)}(0) - \hat{\mathbf{K}}^{(H)} }_{op} \le \frac{\lambda_0}{8}.
\end{align*}
For the other

Recall $\mathbf{A}_{ij}^{(H)} = \begin{pmatrix}
\mathbf{K}^{(H-1)}_{ii} & \mathbf{K}^{(H-1)}_{ij} \\
\mathbf{K}^{(H-1)}_{ji}  & \mathbf{K}^{(H-1)}_{jj}
\end{pmatrix}$ and let $\hat{\mathbf{A}}_{ij}^{(H)} = \begin{pmatrix}
\hat{\mathbf{K}}^{(H-1)}_{ii} & \hat{\mathbf{K}}^{(H-1)}_{ij} \\
\hat{\mathbf{K}}^{(H-1)}_{ji}  & \hat{\mathbf{K}}^{(H-1)}_{jj}
\end{pmatrix}$.

According to Lemma~\ref{lem:22perb} (viewing $\sigma'(\cdot)$ as the $\sigma(\cdot)$ in Lemma~\ref{lem:22perb}), we know \begin{align*}
\abs{\mathbb{E}_{(\vect{U})\sim \hat{\mathbf{A}}_{ij}} \left[\sigma'(u)\sigma'(v)\right] - \mathbb{E}_{(u,v)\sim \mathbf{A}_{ij}} \left[\sigma'(u)\sigma'(v)\right]} \le C\abs{\hat{\mathbf{A}}_{ij}-\mathbf{\mathbf{A}}_{ij}}
\end{align*} for some constant $C>0$.
Since $c$ is small enough, we directly have \begin{align*}
\norm{\hat{\mathbf{K}}^{(H)} -\mathbf{K}^{(H)}}_{op} \le \frac{\lambda_0}{8}
\end{align*}
\end{proof}

\begin{rem}
Combing Theorem~\ref{thm:main_general_framework}, Lemma~\ref{lem:H_concen_mlp}  and standard matrix perturbation bound directly have Lemma~\ref{lem:mlp_least_eigen}.
Similarly we can prove Lemma~\ref{lem:resnet_least_eigen} and Lemma~\ref{lem:conv_resnet_least_eigen}.
\end{rem}

\subsection{Organization}
\label{sec:org}
This paper is organized as follows.
In Section~\ref{sec:rel}, we discuss related works.
In Section~\ref{sec:pre}, we formally state the problem setup.
In Section~\ref{sec:technique}, we present our main analysis techniques.
In Section~\ref{sec:deep_discrete}, we give a warm-up result for the deep fully-connected neural network.
In Section~\ref{sec:resnet}, we give our main result for the ResNet.
In Section~\ref{sec:conv_resnet}, we give our main result for the convolutional ResNet.
We conclude in Section~\ref{sec:conclusion} and defer all proofs to the appendix.

\subsection{Full Rankness of $\mathbf{K}^{(h)}$ for the Fully-connected Neural Network}
\label{sec:fullrank_fc}
In this section we show as long as no two input vectors are parallel, then $\mathbf{K}^{(H)}$ defined in Equation~\eqref{eqn:kernel_resnet} is strictly positive definite.
\begin{prop}\label{prop:fullrank_fc}
Assume $\sigma(\cdot)$ satisfies Condition~\ref{cond:analytic} and for any $i,j \in [n], i\neq j$, $\vect{x}_i \not\parallel \vect{x}_j$.
Then we have $\lambda_{\min}\left(\mat{K}^{(H)}\right) > 0$ where $\lambda_{\min}\left(\mat{K}^{(H)}\right)$ is defined in Equation~\eqref{eqn:kernel_mlp}.
\end{prop}
\begin{proof}[Proof of Proposition~\ref{prop:fullrank_fc}]
By our assumption on the data point and using Lemma~\ref{lem:kernel-positive} we know $\mathbf{K}^{(1)}$ is strictly positive definite.

By letting $\mathbf{Z}= \mathbf{D}^{1/2} \mathbf{U}^\top$ , where $\mathbf{U}\mathbf{D} \mathbf{U}^\top =\mathbf{K}^{h}$. We then use Lemma~\ref{lem:kernel-positive} inductively for $(H-2)$ times to conclude $\mathbf{K}^{(H-1)}$ is strictly positive definite.
Lastly we use Lemma~\ref{lem:gradient-kernel-positive} to finish the proof.
\end{proof}

\begin{lem}\label{lem:kernel-positive}
Assume $\sigma(\cdot)$ is analytic and not a polynomial function.
Consider data $Z = \{\vect z_i \}_{i\in [n]}$ of $n$ non-parallel points (meaning $\vect z_i \notin \text{span}(\vect z_j)$ for all $i \neq j$).
Define \[\mathbf G (Z)_{ij} = \mathbb{E}_{\vect{w}\sim N(\vect{0},\mathbf{I})}[\sigma(\vect w^\top \vect z_i) \sigma(\vect w^\top \vect z_j) ].\] Then $\lambda_{\min} (\mathbf G(Z)) >0$.
\end{lem}
\begin{proof}[Proof of Lemma~\ref{lem:kernel-positive}]
The feature map induced by the kernel $\mathbf G$ is given by $\phi_{\vect z} (\vect w) = \sigma(\vect w^\top \vect z) \vect z$. To show that $\mathbf G(Z)$ is strictly positive definite, we need to show $\phi_{\vect z_1}(\vect w), \ldots, \phi_{\vect z_n} (\vect w)$ are linearly independent functions.
Assume that there are $a_i$ such that
	\begin{align*}
	0 &= \sum_i a_i \phi_{\vect z_i} = \sum_i a_i \sigma( \vect w^\top  \vect z_i ) \vect z_i.
	\end{align*}
	We wish to show that $a_i=0$. Differentiating the above equation $(n-2)$ times with respect to $\vect w$, we have
	\begin{align*}
	0&= \sum_i \left( a_i \sigma^{(n-1)} ( \vect w^\top \vect z_i) \right) \vect  z_i ^{\otimes (n-1)}.
	\end{align*}
Using Lemma~\ref{lem:linear_ind}, we know $\left\{\vect{z}_i^{\otimes (n-1)}\right\}_{i=1}^n$ are linearly independent.
Therefore, we must have  $a_i \sigma^{(n-1)} (\vect  w^\top \vect z_i) = 0$ for all $i$.
Now choosing a $\vect{w}$ such that $\sigma^{(n-1)}\left(\vect{w}^\top \vect{z}_i\right) \neq 0$ for all $i \in [n]$ (such $\vect{w}$ exists because of our assumption on $\sigma$), we have $a_i = 0$ for all $i \in [n]$.
\end{proof}

\begin{lem}\label{lem:gradient-kernel-positive}
Assume $\sigma(\cdot)$ is analytic and not a polynomial function.
Consider data $Z = \{\vect z_i \}_{i\in [n]}$ of $n$ non-parallel points (meaning $\vect z_i \notin \text{span}(\vect z_j)$ for all $i \neq j$).
Define \[\mathbf G(Z)_{ij} = \mathbb{E}_{\vect{w}\sim N(\vect{0},\mathbf{I})}[\sigma'(\vect w^\top \vect z_i) \sigma'(\vect w^\top \vect z_j) (\vect z_i ^\top \vect z_j)].\] Then $\lambda_{\min} (\mathbf G(Z)) >0$.
\end{lem}
\begin{proof}[Proof of Lemma~\ref{lem:gradient-kernel-positive}]
The feature map induced by the kernel $\mathbf G$ is given by $\phi_{\vect z} (\vect w) = \sigma'(\vect w^\top \vect z) \vect z$. To show that $\mathbf G(Z)$ is strictly positive definite, we need to show $\phi_{\vect z_1}(\vect w), \ldots, \phi_{\vect z_n} (\vect w)$ are linearly independent functions.
Assume that there are $a_i$ such that
	\begin{align*}
	0 &= \sum_i a_i \phi_{\vect z_i} = \sum_i a_i \sigma'( \vect w^\top  \vect z_i ) \vect z_i.
	\end{align*}
	We wish to show that $a_i=0$. Differentiating the above equation $(n-2)$ times with respect to $\vect w$, we have
	\begin{align*}
	0&= \sum_i \left( a_i \sigma^{(n)} ( \vect w^\top \vect z_i) \right) \vect  z_i ^{\otimes (n-1)}.
	\end{align*}
Using Lemma~\ref{lem:linear_ind}, we know $\left\{\vect{z}_i^{\otimes (n-1)}\right\}_{i=1}^n$ are linearly independent.
Therefore, we must have  $a_i \sigma^n (\vect  w^\top \vect z_i) = 0$ for all $i$.
Now choosing a $\vect{w}$ such that $\sigma^{(n)}\left(\vect{w}^\top \vect{z}_i\right) \neq 0$ for all $i \in [n]$ (such $\vect{w}$ exists because of our assumption on $\sigma$), we have $a_i = 0$ for all $i \in [n]$.
\end{proof}

\subsection{Full Rankness of $\mathbf{K}^{(h)}$ for ResNet}
\label{sec:fullrank_resnet}

In this section we show as long as no two input vectors are parallel, then $\mathbf{K}^{(H)}$ defined in Equation~\eqref{eqn:kernel_resnet} is strictly positive definite.
Furthermore, $\lambda_{\min}\left(\mat{K}^{(H)}\right)$ does not depend inverse exponentially in $H$.

\begin{prop}\label{prop:H_lambda_0}
	Assume $\sigma(\cdot)$ satisfies Condition~\ref{cond:analytic} and for any $i,j \in [n], i\neq j$, $\vect{x}_i \not\parallel \vect{x}_j$.
	Recall that in Equation~\eqref{eqn:kernel_resnet}, we define \[\mathbf{K}^{(H)}_{ij} =  c_H \mathbf{K}^{(H-1)}_{ij} \cdot \mathbb{E}_{\left(u,v\right)^\top \sim N\left(\vect{0},\begin{pmatrix}
		\mathbf{K}^{(H-1)}_{ii} & \mathbf{K}^{(H-1)}_{ij} \\
		\mathbf{K}^{(H-1)}_{ji} & \mathbf{K}^{(H-1)}_{jj}
		\end{pmatrix}\right) }\left[\sigma'(u)\sigma'(v)\right],
	\]
	where $c_H \sim \frac{1}{H^2}$.
	Then we have $\lambda_{\min} (\mathbf K^{(H)}) \ge c_H \kappa$, where $\kappa$ is a constant that only depends on the activation $\sigma$ and the input data.
In particular, $\kappa$ does not depend on the depth.
\label{prop:resnet-depth-ind}
\end{prop}
\begin{proof}[Proof of Proposition~\ref{prop:H_lambda_0}]
	First note	$\mathbf K_{ii} ^{(H-1)} \in [ 1/c_{x,0}^2, c_{x,0}^2  ]$ for all $H$, so it is in a bounded range that does not depend on the depth (c.f. Lemma~\ref{lem:lem:init_norm_res}).
	Define a function $$\mathbf{G}: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n\times n}$$ such that
$\mathbf{G}(\mathbf K)_{ij} = \mathbf K_{ij} \mathbb{E}_{
	\left(u,v\right)^\top \sim N\left(\vect{0},\begin{pmatrix}
	\mathbf{K}_{ii} & \mathbf{K}_{ij} \\
	\mathbf{K}_{ji} & \mathbf{K}_{jj}
	\end{pmatrix}\right)
	} [ \sigma'(u) \sigma'(v)]$.
	Now define a scalar function \[g(\lambda) = \min_{\mathbf K: \mathbf K \succ 0, \frac{1}{c_{x,0}^2 }\le \mathbf K_{ii}\le c_{x,0 }  , \lambda(\mathbf K) \ge \lambda} \lambda_{\min} (\mathbf G(\mathbf K))  \] with \[\lambda(\mathbf K) = \min_{ij} \begin{pmatrix}
	\mathbf{K}_{ii} & \mathbf{K}_{ij} \\
	\mathbf{K}_{ji} & \mathbf{K}_{jj}
	\end{pmatrix}.\]

	By Lemma~\ref{lem:lambda_lower_bound}, we know
	$\lambda (\mathbf K^{(H-1)}) \ge  c_H \lambda\left(\mathbf{K}^{(0)}\right)$.

	Next, let $\mathbf U\mathbf D\mathbf U^\top =\mathbf K^{(H-1)}$ be the eigen-ecomposition of $\mathbf K$, and $\mathbf Z= \mathbf D^{1/2} \mathbf U^\top$ be the feature embedding into $\mathbb{R}^n$. Since $\begin{pmatrix}
	\vect z_i ^\top \vect z_i & \vect z_i ^\top \vect z_j\\
	\vect z_j ^\top \vect z_i & \vect z_j ^\top \vect z_j
	\end{pmatrix}$  is full rank, then $\vect z_i \notin \text{span}(\vect z_j)$. Then using Lemma \ref{lem:gradient-kernel-positive} , we know $g(\lambda\left(\mathbf{K}^{(0)}\right)) >0$. Thus we have established that $\lambda_{\min} (\mathbf K^{(H)}) \ge c_H g(\lambda\left(\mathbf{K}^{(0)}\right))$ , where $g(\lambda\left(\mathbf{K}^{(0)}\right))$ only depends on the input data and activation $\sigma$. In particular, it is independent of the depth.
\end{proof}

\begin{lem}\label{lem:lambda_lower_bound}
If $\mathcal{D}^{(h)}$ is the identity mapping defined in Section~\ref{sec:general_formulation}, then $\lambda\left(\mathbf{K}^{(H)}\right) \ge \min_{(i,j) \in [n] \times [n]} \lambda_{\min}\left(\begin{array}{ccc}
\mathbf K^{(0)}_{ii} & \mathbf K^{(0)}_{ij}\\
\mathbf K^{(0)}_{ji} & \mathbf K^{(0)}_{jj}\\
\end{array}\right)$.
\end{lem}
\begin{proof}[Proof of Lemma~\ref{lem:lambda_lower_bound}]
First recall

	\begin{equation*}
	(\vect{U},\vect{V})\sim N\left(\vect{0},\left(\begin{array}{ccc}
	\mathcal{W}^{(h)}(\mathbf K_{ii}^{(h-1)}) & \mathcal{W}^{(h)}( \mathbf K_{ij}^{(h-1)})\\
	\mathcal{W}^{(h)}(\mathbf K_{ji}^{(h-1)}) & \mathcal{W}^{(h)}( \mathbf K_{jj}^{(h-1)})\\
	\end{array}\right)\right)
	\end{equation*}
Then we compute
	\begin{equation*}
	\begin{split}
	&\mathbf K^{(h)}_{ij}-\vect b^{(h)}_i\vect b^{(h)\top}_j\\
	&=\mathcal{D}^{(h)}\mathbf K^{(h-1)}_{ij}\mathcal{D}^{(h)\top}+\mathbb{E}_{(\vect U,\vect V)}\Big(\rho(\vect U) \mathcal{D}^{(h)}(\vect b^{(h-1)}_j)^\top+(\mathcal{D}^{(h)}(\vect b^{(h-1)}_i)) \rho(\vect V)^\top+\rho(\vect U)\rho(\vect V)^\top)\Big)\\
	&-\left(\mathcal{D}^{(h)}(\vect b_i^{(h-1)})+
	\mathbb{E}_{\vect U}\rho^{(h)}(\vect U)\right)\left(\mathcal{D}^{(h)}(\vect b_j^{(h-1)})+\mathbb{E}_{\vect V}\rho^{(h)}(\vect V))\right)^\top\\
	&=\mathcal{D}^{(h)}\left(\mathbf K^{(h-1)}_{ij} - \vect b_i^{(h-1)}\vect b_j^{(h-1)\top}\right)\mathcal{D}^{(h)\top}+\mathbb{E}_{(\vect U,\vect V)}\Big(\rho(\vect U)\rho(\vect V)^\top)\Big)-\left(\mathbb{E}_{\vect U}\rho^{(h)}(\vect U)\right)\left(\mathbb{E}_{\vect V}\rho^{(h)}(\vect V))\right)^\top\\
	\end{split}
	\end{equation*}

	For ResNet, $\mathcal{D}^{(h)}$ is the identity mapping so
we have
	\begin{equation*}
	\begin{split}
	&\mathbf K^{(h)}_{ij}-\vect b^{(h)}_i\vect b^{(h)\top}_j\\
	&=\mathbf K^{(h-1)}_{ij}-\vect b_i^{(h-1)}\vect b_j^{(h-1)\top}+\mathbb{E}_{(\vect U,\vect V)}\Big(\rho(\vect U)\rho(\vect V)^\top)\Big)-\left(\mathbb{E}_{\vect U}\rho^{(h)}(\vect U)\right)\left(\mathbb{E}_{\vect V}\rho^{(h)}(\vect V))\right)^\top.
	\end{split}
	\end{equation*}

To proceed, we calculate

	\begin{equation*}
	\begin{split}
	&\left(\begin{array}{ccc}
	\mathbf K^{(h)}_{ii} & \mathbf K^{(h)}_{ij} \\
	\mathbf K^{(h)}_{ji} & \mathbf K^{(h)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(h)}\\
	\vect b_j^{(h)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(h)\top},
	\vect b_j^{(h)\top} \\
	\end{array}\right)\\
	=&\left(\begin{array}{ccc}
	\mathbf K^{(h-1)}_{ii} & \mathbf K^{(h-1)}_{ij} \\
	\mathbf K^{(h-1)}_{ji} & \mathbf K^{(h-1)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(h-1)}\\
	\vect b_j^{(h-1)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(h-1)\top},
	\vect b_j^{(h-1)\top} \\
	\end{array}\right)\\
&+ \left(\mathbb{E}_{\vect U,\vect V}\left(\begin{array}{ccc}
	\rho^{(h)}(\vect U)\rho^{(h)}(\vect U)^{\top} & \rho^{(h)}(\vect U)\rho^{(h)}(\vect V)^{\top} \\
	\rho^{(h)}(\vect V)\rho^{(h)}(\vect U)^{\top} & \rho^{(h)}(\vect V)\rho^{(h)}(\vect V)^{\top}\\
	\end{array}\right)-
	\mathbb{E}_{\vect U,\vect V}\left(\begin{array}{ccc}
	\rho(\vect U)\\
	\rho(\vect V) \\
	\end{array}\right)\mathbb{E}_{\vect U,\vect V}\left(\begin{array}{ccc}
	\rho(\vect U)^{\top},
	\rho(\vect V)^{\top} \\
	\end{array}\right)\right)\\
\ge&\left(\begin{array}{ccc}
	\mathbf K^{(h-1)}_{ii} & \mathbf K^{(h-1)}_{ij} \\
	\mathbf K^{(h-1)}_{ji}  & \mathbf K^{(h-1)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(h-1)}\\
	\vect b_j^{(h-1)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(h-1)\top},
	\vect b_j^{(h-1)\top} \\
	\end{array}\right)
	\end{split}
	\end{equation*}

	As a result, we have

	\begin{equation}
	\begin{split}
	&\lambda_{\min}\left(\begin{array}{ccc}
	\mathbf K^{(h)}_{ii} & \mathbf K^{(h)}_{ij} \\
	\mathbf K^{(h)}_{ji} & \mathbf K^{(h)}_{jj}\\
	\end{array}\right)\\
	\ge&\lambda_{\min}\left(\begin{array}{ccc}
	\mathbf K^{(h)}_{ii} & \mathbf K^{(h)}_{ij} \\
	\mathbf K^{(h)}_{ji} & \mathbf K^{(h)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(h)}\\
	\vect b_j^{(h)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(h)\top},
	\vect b_j^{(h)\top} \\
	\end{array}\right)\\
	\ge&\min \lambda_{\min}\left(\begin{array}{ccc}
	\mathbf K^{(h-1)}_{ii} & \mathbf K^{(h-1)}_{ij} \\
	\mathbf K^{(h-1)}_{ji} & \mathbf K^{(h-1)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(h-1)}\\
	\vect b_j^{(h-1)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(h-1)\top},
	\vect b_j^{(h-1)\top} \\
	\end{array}\right)\\
	\ge &\cdots\\
	\ge&\lambda_{\min}\left(\begin{array}{ccc}
	\mathbf K^{(0)}_{ii} & \mathbf K^{(0)}_{ij} \\
	\mathbf K^{(0)}_{ji} & \mathbf K^{(0)}_{jj}\\
	\end{array}\right)-\left(\begin{array}{ccc}
	\vect b_i^{(0)}\\
	\vect b_j^{(0)} \\
	\end{array}\right)\left(\begin{array}{ccc}
	\vect b_i^{(0)\top},
	\vect b_j^{(0)\top} \\
	\end{array}\right) \\
	= &\lambda_{\min}\left(\begin{array}{ccc}
	\mathbf K^{(0)}_{ii} & \mathbf K^{(0)}_{ij} \\
	\mathbf K^{(0)}_{ji} & \mathbf K^{(0)}_{jj}\\
	\end{array}\right).
	\end{split}
	\label{eq:non-decreasing}
	\end{equation}
We now prove the theorem.
\end{proof}

\section{Introduction}
\label{sec:intro}
\input{intro.tex}

\section{Related Works}
\label{sec:rel}
\input{rel.tex}

\section{Preliminaries}
\label{sec:pre}
\input{pre.tex}

\section{Technique Overview}
\label{sec:technique}
\input{technique.tex}

\section{Warm Up: Convergence Result of GD for Deep Fully-connected Neural Networks}
\label{sec:deep_discrete}
\input{deep_discrete.tex}

\section{Convergence Result of GD for ResNet}
\label{sec:resnet}
\input{resnet.tex}

\section{Convergence Result of GD for Convolutional ResNet}
\label{sec:conv_resnet}
\input{conv_resnet.tex}

\section{Conclusion}
\label{sec:conclusion}
\input{conclusion.tex}

\section*{Acknowledgments}
\input{ack.tex}

\subsection{Notations}
We Let $[n] = \{1, 2, \ldots, n\}$.
We use $N(\vect{0},\mathbf{I})$ to denote the standard Gaussian distribution.
For a matrix $\vect A$, we use $\mathbf A_{ij}$ to denote its $(i, j)$-th entry. We will also use $\mathbf{A}_{i,:}$ to denote the $i$-th row vector of $\mathbf{A}$ and define $\mathbf{A}_{i,j:k}=(\mathbf{A}_{i,j},\mathbf{A}_{i,j+1},\cdots,\mathbf{A}_{i,k})$ as part of the vector.
Similarly $\mathbf{A}_{:,i}$ is the $i$-th column vector and $\mathbf{A}_{j:k,i}$ is a part of $i$-th column vector.
For a vector $\vect{v}$, we use $\norm{\vect{v}}_2$ to denote the Euclidean norm.
For a matrix $\mathbf{A}$ we use $\norm{\mathbf{A}}_F$ to denote the Frobenius norm and $\norm{\mathbf{A}}_2$ to denote the operator norm.
If a matrix $\mathbf{A}$ is positive semi-definite, we use $\lambda_{\min}(\mathbf{A})$ to denote its smallest eigenvalue.
We use $\langle \cdot, \cdot \rangle$ to denote the standard Euclidean inner product between two vectors or matrices.
We let $O(\cdot)$ and $\Omega\left(\cdot\right)$ denote standard Big-O and Big-Omega notations, only hiding  constants.
In this paper we will use  $C$ and $c$ to denote constants. The specific value can be different from line to line.

\subsection{Activation Function}
We use $\relu{\cdot}$ to denote the activation function.
In this paper we impose some technical conditions on the activation function.
The guiding example is softplus: $\relu{z} = \log(1+\exp(z))$.
\begin{condition}[Lipschitz and Smooth]\label{cond:lip_and_smooth}
There exists a constant $c>0$ such that	$\abs{\relu{0}} \le c$ and for any $z,z' \in \mathbb{R}$,
\begin{align*}
\abs{\relu{z} -\relu{z'} }\le &c\abs{z-z'}, \\
\text{ and }\abs{\sigma'(z)-\sigma'(z)} \le &c \abs{z-z'}.
\end{align*}
\end{condition}
These two conditions will be used to show the stability of the training process.
Note for softplus both Lipschitz constant and smoothness constant are $1$.
In this paper, we view all activation function related parameters as constants.

\begin{condition}\label{cond:analytic}
$\relu{\cdot}$ is analytic and is not a polynomial function.
\end{condition}
This assumption is used to guarantee the positive-definiteness of certain Gram matrices which we will define later.
Softplus function satisfies this assumption by definition.

\subsection{Problem Setup}
In this paper, we focus on the empirical risk minimization problem with the quadratic loss function\begin{align}
	\min_{\vect{\theta}} L(\vect{\theta}) = \frac{1}{2}\sum_{i=1}^{n}(f(\vect{\theta},\vect{x}_i)-y_i)^2 \label{eqn:loss}
\end{align}where $\left\{\vect{x}_i\right\}_{i=1}^n$ are the training inputs, $\left\{y_i\right\}_{i=1}^n$ are the labels, $\vect{\theta}$ is the parameter we optimize over and $f$ is the prediction function, which in our case is a neural network.
We consider the following architectures.
\begin{itemize}
\item \textbf{Multilayer fully-connected neural networks:}
Let $\vect{x} \in \mathbb{R}^{d}$ be the input, $\mathbf{W}^{(1)} \in \mathbb{R}^{m \times d}$ is the first weight matrix, $\mathbf{W}^{(h)} \in \mathbb{R}^{m \times m}$ is the weight at the $h$-th layer for $2\le h\le H$, $\vect{a} \in \mathbb{R}^{m}$ is the output layer and $\relu{\cdot}$ is the activation function.\footnote{We assume intermediate layers are square matrices for simplicity. It is not difficult to generalize our analysis to rectangular weight matrices.}
We define the prediction function recursively (for simplicity we let $\vect{x}^{(0)}=\vect{x}$).
\begin{align}
\vect{x}^{(h)}&=\sqrt{\frac{c_{\sigma}}{m}} \relu{\mathbf{W}^{(h)}\vect{x}^{(h-1)}}, 1\le h \le H\nonumber\\
f(\vect{x},\vect{\theta})&=\vect{a}^\top \vect{x}^{(H)}.
\label{eqn:mlp}
\end{align}
where
$c_{\sigma}=\left(\mathbb{E}_{x\sim N(0,1)}\left[\sigma(x)^2\right]\right)^{-1}$ is a scaling factor to normalize the input in the initialization phase.

\item \textbf{ResNet}\footnote{We will refer to this architecture as ResNet, although this differs by the standard ResNet architecture since the skip-connections at every layer, instead of every two layers.
This architecture was previously studied in \cite{hardt2016identity}.
We study this architecture for the ease of presentation and analysis.
It is not hard to generalize our analysis to architectures with skip-connections are every two or more layers.
}:
We use the same notations as the multilayer fully connected neural networks.
We define the prediction recursively.
\begin{align}
\vect{x}^{(1)}&=\sqrt{\frac{c_{\sigma}}{m}} \relu{\mathbf{W}^{(1)}\vect{x}}, \nonumber\\
\vect{x}^{(h)} & =\vect{x}^{(h-1)}+\frac{c_{res}}{H\sqrt{m}} \relu{\mathbf{W}^{(h)}\vect{x}^{(h-1)}} \nonumber\\
& ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\text{ for } 2\le h\le H, \nonumber\\
f_{res}(\vect{x},\vect{\theta})&=\vect{a}^\top \vect{x}^{(H)}
 \label{eqn:resnet}
\end{align}
where  $0< c_{res} < 1$ is a small constant.
Note here we use a $\frac{c_{res}}{H\sqrt{m}}$ scaling. This scaling plays an important role in guaranteeing the width per layer only needs to scale polynomially with $H$.
In practice, the small scaling is enforced by a small initialization of the residual connection~\citep{hardt2016identity,zeroinit2018}, which obtains state-of-the-art performance for deep residual networks. We choose to use an explicit scaling, instead of altering the initialization scheme for notational convenience.

\item \textbf{Convolutional ResNet}:
Lastly, we consider the convolutional ResNet architecture.
Again we define the prediction function in a recursive way.

	Let $\vect{x}^{(0)} \in \mathbb{R}^{d_0\times p}$ be the input, where $d_0$ is the number of input channels and $p$ is the number of pixels.
	For $h \in [H]$, we let the number of channels be $d_h = m$ and number of pixels be $p$.
	Given $\vect{x}^{(h-1)} \in \mathbb{R}^{d_{h-1}\times p}$ for $h \in [H]$,
	we first use an operator $\phi_h(\cdot)$ to divide $\vect{x}^{(h-1)} $ into $p$ patches.
	Each patch has size $qd_{h-1}$ and this  implies a map $\phi_h(\vect{x}^{(h-1)})\in\mathbb{R}^{qd_{h-1}\times p}$.
	For example, when the stride is $1$ and $q=3$
	\begin{align*}
	&\phi_h(\mathbf{x}^{(h-1)}) \\
	=&\begin{pmatrix}
	\left(\mathbf{x}^{(h-1)}_{1,0:2}\right)^\top,
	&\ldots &, \left(\mathbf{x}^{(h-1)}_{1,p-1:p+1}\right)^\top\\
	\ldots,
	& \ldots, & \ldots \\
	\left(\mathbf{x}^{(h-1)}_{d_{h-1},0:2}\right)^\top,
	&\ldots, &  \left(\mathbf{x}^{(h-1)}_{d_{h-1},p-1:p+1}\right)^\top
	\end{pmatrix}
	\end{align*}

where we let $\mathbf{x}^{(h-1)}_{:,0}=\mathbf{x}^{(h-1)}_{:,p+1}=\vect{0}$, i.e., zero-padding.
Note this operator has the property\begin{align*}
 \norm{\vect{x}^{(h-1)}}_F \le \norm{\phi_h(\vect{x}^{(h-1)})}_F \le \sqrt{q}\norm{\vect{x}^{(h-1)}}_F .
	\end{align*}
because each element from $\vect{x}^{(h-1)}$ at least appears once and at most appears $q$ times.
In practice, $q$ is often small like $3\times 3$, so throughout the paper we view $q$ as a constant in our theoretical analysis.
To proceed, let $\mathbf{W}^{(h)} \in \mathbb{R}^{d_h \times qd_{h-1}}$, we have \begin{align*}
	\vect{x}^{(1)}  =&\sqrt{\frac{c_{\sigma}}{m}} \relu{\mathbf{W}^{(1)} \phi_1(\vect{x})} \in \mathbb{R}^{m\times p},\\
		\vect{x}^{(h)}  =&\vect{x}^{(h-1)}+\frac{c_{res}}{H\sqrt{m}} \relu{\mathbf{W}^{(h)} \phi_h(\vect{x}^{(h-1))}} \in \mathbb{R}^{m\times p} \\
&~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~		\text{for }2\le h\le H,
	\end{align*} where  $0< c_{res} < 1$ is a small constant.
	Finally, for $\vect{a} \in \mathbb{R}^{m \times p}$, the output is defined as \begin{align*}
	f_{cnn}(\vect{x},\vect{\theta}) = \langle \vect{a}, \vect{x}^{(H)} \rangle.
	\end{align*}
	Note here we use the similar scaling $O(\frac{1}{H\sqrt{m}})$ as ResNet.

\end{itemize}

To learn the deep neural network, we consider the randomly initialized gradient descent algorithm to find the global minimizer of the empirical loss~\eqref{eqn:loss}.
Specifically, we use the following random initialization scheme.
For every level $h \in [H]$, each entry is sampled from a standard Gaussian distribution, $\mathbf{W}_{ij}^{(h)} \sim N(0,1)$ and each entry of the output layer $\vect{a}$ is also sampled from $N(0,1)$.
In this paper, we train all layers by gradient descent, for $k=1,2,\ldots,$ and $h\in[H]$\begin{align*}
\mathbf{W}^{(h)}(k) &= \mathbf{W}^{(h)}(k-1) - \eta \frac{\partial L(\vect{\theta}(k-1))}{\partial \mathbf{W}^{(h)}(k-1)},\\
\vect{a}(k) &= \vect{a}(k-1) - \eta \frac{\partial L(\vect{\theta}(k-1))}{\partial \vect{a}(k-1)}
\end{align*} where $\eta >0 $ is the step size.

redpajama_set_name: RedPajamaArXiv


text:
import { Source, LayerType } from '@vizzuality/layer-manager';
import type { GeoJSONSourceRaw } from 'mapbox-gl';
import type { TileLayer } from '@deck.gl/geo-layers';

export interface MaskLayer extends GeoJSONSourceRaw {
  id: string;
  opacity?: number;
}

export interface Render {
  layers?: Record<string, string | number | boolean | unknown>[];
}

export interface layerConfigBodySpec {
  attribution?: string;
}

export interface DeckProps {
  type: 'deck';
  params?: Record<string, unknown>;
  data: string;
  subtype: string;
}

export interface layerConfigSpec {
  type: LayerType;
  render?: Render;
  source: Partial<Source>;
  [key: string]: Record<string, string | number | boolean | unknown> | string | boolean | number;
  attribution?: string;
  body?: layerConfigBodySpec;
  deck?: DeckProps;
}

export interface APILayerSpec {
  id: string;
  name: string;
  dataset: string;
  slug: string;
  description?: string;
  application: string[];
  type?: string;
  userId: string;
  iso: string[];
  provider: string;
  userId: string;
  default: boolean;
  protected: boolean;
  published: boolean;
  env: string;
  thumbnailUrl: string;
  layerConfig: layerConfigSpec;
  legendConfig: Record<string, string | number | boolean | unknown>;
  applicationConfig: Record<string, string | number | boolean | unknown>;
  interactionConfig: Record<string, string | number | boolean | unknown>;
  staticImageConfig: Record<string, string | number | boolean | unknown>;
  createdAt: string;
  updatedAt: string;
}

export interface DeckLayerSpec {
  id: string;
  type: 'deck';
  deck: TileLayer[];
}

redpajama_set_name: RedPajamaGithub


Hello everybody and happy Thursday.
Sugar. Such a sweet thing that is so bad for you. Natural sugars – like those found in fruits, vegetables, dairy and grain – are easily processed by your body to be a slow and steady supply of energy. But added sugars – processed sugars – are not processed well by your body. They spike insulin in your blood and then stored in your body as fat.
I will not deny that processed sugars taste good. They do. Candies, cakes, soda, pies, etc. are things that we have eaten for rewards, special occasions and as emotional support. But that does not make them good for us. Our bodies are screaming at us for stuffing that bad food into our bodies. We are forcing our liver to handle things it was not designed to handle. So the liver does the only thing it knows to do with processed sugar (fructose), it makes fat cells with it. Actually, according to WebMD.com, there are 12 ways sugar harms out bodies.
Weight gain. Putting on too much weight can cause diabetes and some cancers.
Heart Disease. One in 10 Americans gets 1/4 or more of their daily calories from added sugar. If you are one of those people you are more than twice as likely to die from heart disease.
Diabetes. When sugar stays in your bloodstream for long periods, your body has a tendency to make less insulin, or the insulin does not work as well.
High blood pressure. There is research that shows sugar raises blood pressure by spiking the insulin level too high. This causes the blood vessels to be less flexible and cause your kidneys to hold onto water and sodium.
High cholesterol. Sugary diets raise your LDL (bad cholesterol) and lowers the HDL (good cholesterol). Sugary diets also raise the blood fats, triglycerides.
Liver disease. Processed sugar is called fructose. Fructose is turned into fat in your liver. This fat builds up in your liver and causes swelling and scarring.
Cavities. Sugar causes cavities by feeding the bad bacteria in your mouth.
Poor sleep. Sugar messes with your blood glucose levels causing energy spikes and crashes.
ADHD. This is added as a possible harm. Research is ongoing about this. But any parent can tell you that kids get pretty hyper after feasting on Easter candy and Halloween candy. Am I right?
Mood problems. Sugar has been linked to mental health problems. Sugar can fuel depression and anxiety.
Gout. You have heard that gout is caused by eating too much red meat but it also is caused by eating too much organ meats, lobster AND SUGAR.
Kidney stones. Too much fructose (processed sugar) causes kidney stones.
Sugar is not the sweet little treat that we imagine it is. Granted a bite or two every once in a while will not cause the terrible consequences listed above. But how many of us can stop at just a bite or two?
Whether or not you eat processed sugar is your business. As so me, I am going to cut it out of my diet as much as possible. My rule of thumb is, if it's not actually home-made, I'm not going to eat it. That means if my Mother makes one of her awesome chocolate pies, I'm going to have a slice. Then I will leave the rest of it alone – no matter how much she tries to make me eat it. Now, if all she has is a store bought pie (ex: Mrs. Edwards), I am not going to eat that. Too many preservatives and ingredients I don't want in my body.


text:
<!DOCTYPE html>

	<meta charset="utf-8">

	<title>JMSGGroupInfo Class Reference</title>

	<link rel="stylesheet" href="../css/style.css">
	<meta name="viewport" content="initial-scale=1, maximum-scale=1.4">
	<meta name="generator" content="appledoc 2.2.1 (build 1334)">

				JMessage-iOS-SDK

				HXHG

Index

Hierarchy

							On This Page

							<select id="jump-to">
	<option value="top">Jump To…</option>

	<option value="overview">Overview</option>

	<option value="tasks">Tasks</option>

	<optgroup label="Properties">

		<option value="//api/name/gid">gid</option>

		<option value="//api/name/name">name</option>

		<option value="//api/name/desc">desc</option>

		<option value="//api/name/avatarData">avatarData</option>

		<option value="//api/name/avatar">avatar</option>

		<option value="//api/name/groupType">groupType</option>

		<option value="//api/name/maxMemberCount">maxMemberCount</option>

	</optgroup>

</select>

JMSGGroupInfo Class Reference

	Inherits from
	NSObject

	Declared in
	JMSGGroup.h

Overview

群信息类(此类仅用于修改群信息、创建群、群信息展示)

注意:

如果想要获取群的相关属性值、调用相关接口,需要通过 gid 获取到 JMSGGroup 对象再使用;

本类中可读可写属性表示可以用于群信息修改、创建群传值,只读属性说明是不允许客户端修改的,只做展示;

  gid

群 id

@property (nonatomic, strong, readonly) NSString *gid

Declared In

JMSGGroup.h

  name

群名称

@property (nonatomic, strong, readwrite) NSString *name

Declared In

JMSGGroup.h

  desc

群描述

@property (nonatomic, strong, readwrite) NSString *desc

Declared In

JMSGGroup.h

  avatarData

群头像数据,此属性只用户修改群信息,切勿从此类拿来此属性来展示

@property (nonatomic, strong, readwrite) NSData *avatarData

Declared In

JMSGGroup.h

  avatar

群头像的媒体文件ID

@property (nonatomic, strong, readonly) NSString *avatar

Declared In

JMSGGroup.h

  groupType

群组类型,私有、公开,注意:仅限于创建群组时设置,创建成功之后不允许修改群类型

@property (nonatomic, assign, readwrite) JMSGGroupType groupType

Declared In

JMSGGroup.h

  maxMemberCount

群组人数上限

@property (nonatomic, strong, readonly) NSString *maxMemberCount

Declared In

JMSGGroup.h

Copyright © 2018 HXHG. All rights reserved. Updated: 2018-03-01

Generated by appledoc 2.2.1 (build 1334).


text:
Венецуела су се први пут појавила на Олимпијским играма 1948. године и слала је своје спортисте на све касније одржане Летњих олимијских игара.

Венецуела је први пут на Зимске олимпијске игре послала своје представнике 1998. године. Представници Венецуеле, закључно са Олимпијским играма одржаним 2008. године у Пекингу, су освојили 11 олимпијских медаља а од тога једну златну на играма одржаним 1968. године у Мексико Ситију.

Национални олимпијски комитет Венецуеле (-{Comité Olímpico Venezolano}-) је основан 1935. и признат од стране МОКа исте године.

Медаље

Летње олимпијске игре

Освојене медаље на ЛОИ

Освајачи медаља на ЛОИ

Зимске олимпијске игре

Освојене медаље на ЗОИ

Преглед учешћа спортиста Венецуеле по спортовима на ЗОИ
После ЗОИ 2010.

Разлика у горње две табеле у 2 учесника 1 мушкарац и 1 жена настала је у овој табели јер је сваки спотиста без обриза колико је пута учествовао на играма и у колико разних спортова на истим играма рачунат само једном

Укупно медаље на ОИ
После ЗОИ 2010.

Занимљивости

Спољашње везе

 Освајачи олимпијских медаља

Венецуела на олимпијским играма


The Illustration Work of Louise Byng as seen from Earth.: Let's Draw.
I woke up on 8th Jan to hear the world whispering about David Bowie. Why isn't every day like this? I thought, to find the tweets and articles had been triggered by Bowie's return to releasing tunes at the age of 66. Needless to say this prompted a drawing, hoping to capture his chiseled, now aging, face as well as the colourful character(s) to be found inside him.
Super cool, and now everyone's talking about him.