text:
The Oklahoma City Museum of Art will launch "The Painters of Pompeii" on June 26
A number of collection highlights will travel to North America for the first time as part of the exhibition The Painters of Pompeii: Roman Frescoes from the National Archaeological Museum, Naples. This landmark exhibition on the art of painting in ancient Rome, which opens on June 26 and runs through October 17, will be shown exclusively at the Oklahoma City Museum of Art before traveling to Europe.
Beginning June 26, visitors to OKCMOA will be taken to ancient Rome by magnificent wall paintings, or frescoes, from Pompeii and Herculaneum. "The Painters of Pompeii: Roman Frescoes from the National Archaeological Museum, Naples" explores the seldom-seen world of ancient Roman painters, revealing insight into their methods, materials, and techniques via over 80 objects and artworks.
"In 79 CE, Mount Vesuvius erupted, covering the seaside resort towns of Pompeii and Herculaneum in ash and volcanic rock," said Dr. Michael J. Anderson, OKCMOA president and CEO. "The eruption preserved Roman luxury villas and homes until excavations began in the 1700s. These excavations unearthed extensive and richly colored frescoes painted on the walls of homes and public buildings throughout the cities."
Archivio Fotografico del Museo Archeologico Nazionale di Napoli
"Early in the excavations, many of the paintings were cut out of the walls and taken to museums," Anderson continued. "It is incredibly rare for ancient paintings to survive into the present day, much less to be taken to a museum for display. We are thrilled to provide the incredible opportunity to be immersed in the day-to-day lives of the ancient Romans."
"The Painters of Pompeii" highlights a seldom-seen medium–the Roman wall painting, which was prevalent in ancient Rome. The interiors of Roman buildings were sumptuously decorated with bold colors and designs that ranged in theme from mythology to landscapes to still lifes to architecture. Often, these paintings covered the entire wall, from floor to ceiling, and transported the viewer beyond the confines of the room.
The exhibition is free for Museum members, kids 17 and under, and active military as part of Blue Star Museums; tickets ($16 for adults) are on sale now. Free children's admission is possible thanks to generous support from The Chickasaw Nation, The Ann Lacy Foundation, Cresap Family Foundation, and Chuck and Renate Wiggin. Pre-sales are strongly encouraged for this high-demand North American exclusive.
This exhibition is organized by the National Archaeological Museum, Naples, and MondoMostre and is presented by George Records and the Inasmuch Foundation.
Source: OKCMOA
Cover Photo: the National Archaeological Museum, Naples
By Leman Altuntaş 23 June 2021
Tags: Ancient Rome, Archivio Fotografico del Museo Archeologico Nazionale di Napoli, Blue Star Museums, Featured, Naples, OKCMOA, the National Archaeological Museum, the Oklahoma City Museum, The Painters of Pompeii in Classical Archeology, News
Mystery ax discovered off the coast of Arendal of Norway
Researchers have discovered a find that could be a first for Norwegian archeology. A hollow ax, which researchers believe dates...
Classical Archeology
The Sad Story of Hypatia of Alexandria
Hypatia of Alexandria was one of the few women in ancient Greek academia. Most of all people remember Hypatia of...
İnteresting Relief on the Roman Millstone
During the Cambridgeshire A14 road improvement work, workers found an interesting millstone. A large penis was engraved in the Roman-era...
1500-year-old Medallion Rescued From Treasure Hunters on Display in Çorum Museum
A 1,500-year-old gold medallion portraying a figure of Jesus Christ has been exhibited at a museum in Turkey's northern province...
Return of a 4,250-year-old Hattian golden beak-spouted ewer to Turkey
The 4,250-year-old golden beak-spouted ewer was returned to the Anatolian Civilizations Museum by the Gilbert Art Foundation. Culture and Tourism...
Anatolia's largest olive oil factory unearthed
A Roman-era olive oil factory has been unearthed during excavations in the İskenderun district of Hatay. It has been reported...
The ancient city of Kastabala will soon have a colonnaded Street
The archaeological excavation of the ancient city of Kastabala in Osmaniye Province in southern Turkey continues. Kastabala-Hierapolis is one of...
Interesting News, News
'Dinosaur dance floor' dating back 80 million years found in China
In China, researchers have found many dinosaur footprints in an area of 1,600 square meters described in the literature as...
Maltaş Temple Revealed
Phrygian Valley, 10 meters high monument with Phrygian scriptures inscriptions on it discovered. The unearthed Maltaş monument is actually the...
In the city of Gods and Goddesses Magnesia, Zeus Temple's entrance gate found
During an excavation in the ancient city of Magnesia, located in the Ortaklar district of Germencik in Turkey's Aegean province...
Researchers Finds Nearly 500 Ancient Ceremonial Sites in Southern Mexico with Lidar Technique
A team of international researchers led by the University of Arizona reported last year that they had uncovered the largest...
Anatolian Archeology, News, Travel
3600 years old Unique ancient drinking bowls on display at Boğazkale Museum
The 3,600-year-old fist-shaped drinking bowls found in excavations in Hattusa, the capital of the Hittite Civilization, which shaped the Anatolian...
In Bergama, the City of Greek Gods, the People Kept the Cult of Cybele Alive
The figurines of Cybele, the goddess of the fertility of Anatolia, and the presence of sanctuaries unearthed in the Ancient...
Maya Farmers May Have Planned Population Growth Contrary to Thought
Contrary to what was thought, Maya farmers may have planned for population growth, says a new study. According to a...


text:
15 Rare Photos Of WWE Women With Baby Bumps
by Alexander Keobke
in Wrestling
One of the most magical experiences that a person can have is becoming a parent. It's an experience that many of the best women in WWE history have had on multiple occasions. Even if it's an experience that in many instances, the women definitely were not planning to engage in. This includes Maryse and Mike Mizanin who were left blindsided with the knowledge that they were going to become parents. Mickie James and Maria Kanellis also weren't expecting to add to their family, but they've definitely made the best of it. And while Christy Hemme may have planned to get pregnant for a 2nd time with her husband, there is no way that she was expecting to be pregnant with quadruplets. Though if the 4 of them ever have a feud they can pair off into tag teams and have her other child step in as the referee!
But at least Maria Kanellis and Maryse received proper support from the company when they announced their news, including taking maternity leave. This is something Vince may have made sure of, as when Dawn Marie was fired when she was still expecting, she handed the WWE a lawsuit for wrongful termination that they definitely weren't expecting!
While you may be used to seeing these women flash their toned abs in the ring - or in some cases, show a little more than that outside of it - that's not what you'll be seeing below. Instead, we've tracked down 15 of the rarest photos of WWE Women when they were still eating for two.
15 Stacy Keibler
via wp.com
If you were thinking of what Stacy Keibler's most popular relationship was, it may be when she won over the affections of the former Batman, George Clooney. I mean, sure, he was like the worst Batman, but Clooney still can say he's been the caped crusader. But while the two spent several years together, the two never tied the knot. That wasn't the case for the next man that Keibler won over, Jared Pobre, who had previously been friends with Keibler. Do you think Clooney was ever jealous of how close the two seemed?
The two have been married since 2014 and as you can tell from this photo, have started a family together.
Keibler gave birth to a daughter, Ava, back in 2014. While many women elect to have their child in the hospital and with the assistance of various drugs, that wasn't the case for Keibler "When I was pregnant, I did concurrent care with both my OB-gyn and midwife because I wanted to compare the journey. I concluded that there was no doubt that I wanted to have a home birth with no medication." She may have had second thoughts about that plan after her labor lasted 19 hours though! (Source: EOnline).
14 Eve Torres
via imgur.com
One of the most difficult things for a woman to try and achieve is to adjust to their body after giving birth. When you consider WWE athletes live their life under the public eye and are constantly putting their body through the ringer, that pressure may be even more for them to overcome. When discussing with US Weekly how she felt during her own pregnancy, as well as her feelings after giving birth to her son, Raeven, in 2015 she stated "As someone whose profession relies heavily on physical activity, I definitely felt extra pressure to get back to my pre-baby shape. But the truth is, I don't know a mom who doesn't feel the pressure to bounce back after giving birth."
If that wasn't inspiring enough, she also went on to state "Exercise (when your body is ready), eat for health, love yourself & your child, and let your body do whatever it's meant to do after the amazing feat of creating a human." We are sure there are many women out there who need to read inspirational things like this from women they respect like Torres. Credit also goes to Torres' husband Rener Gracie, for clearly creating an environment where she feels loved and supported.
13 Christy Hemme
via blogspot.com
Christy Hemme definitely had to beat some pretty unlikely odds to come out on top of the 2004 WWE Diva Search. But credit where credit is due, Hemme did have a reasonably long career in the industry before officially retiring in 2009. But if you want even crazier odds, you won't believe this next entry. Christy Hemme and her husband Charley Patterson who got married in 2010 first got pregnant and gave birth to a girl, Charlie Rose who she gave birth to on January 6th, 2015.
When talking about her first pregnancy, Hemme stated "We have always known we wanted to have kids, but we also wanted to have at least five years for us to travel and enjoy each other. We have now been together for six years and really feel like we took the time to create a solid foundation for children; a foundation that will show them we are committed to one another and to them. We want to show them each day what true love looks like." In September 2017, Hemme and Patterson announced they were pregnant but this time with quadruplets. That's still not the crazy part. When Hemme eventually gave birth - to 3 boys and 1 girl - IT WAS ALSO ON JANUARY 6th. As if that birthday party wasn't big enough for the 4 of them? (Source: FitPregnancy).
12 Maryse
via eonline.com
WWE Wrestlers put themselves in dangerous situations on a regular basis where they always need to be sure they're being safe. But for Maryse and Mike Mizanin, their safety should have perhaps continued on when they engaged in cardio that resulted in a little Miz. The Miz was clearly flabbergasted with the news (as he recalled they weren't trying for a child) when Maryse not only let him know she was pregnant but did so while filming an episode for Total Divas. But hey, it makes good TV right?
At least the two were in a loving relationship and already married at the time of the revelation or the news may have been a lot more awkward!
When describing her first moments with her daughter, Monroe Sky, who was born back in March, Maryse stated "I will never forget the first little noise she made. Hearing her … that sound of her tiny cry will never leave my memory. This little girl is everything." The Miz also had a touching story about when he first truly felt like he was a Dad, "It became real for me at about 2 a.m., when Monroe was crying and I picked her up, put her in my arms and rocked her in the rocking chair. I was so tired, but she stopped crying and just stared at me with those beautiful blue eyes that melted my heart. I love her so much." (Source: People, People (2)).
11 Rosa Mendes
Rosa Mendes may not have put together a career worthy of a spot in the Hall of Fame, but she still achieved her dream of being in the WWE. But when she sat down with the company to talk about why she was stepping away from the ring, she talked about her daughter that came from the bump you see above, "I just feel like as you get older, your dreams evolve. For years, my dream was to become a WWE champion. And, now that I've brought a life into this world, and I'm with the man of my dreams, it's different." The man that Mendes is talking about is Bobby Schubenski, who Mendes has been engaged to since 2015. Mendes gave birth to her daughter, who she named Jordan Elizabeth, back in February 2016. While we're sure they've taken some steps towards marriage, perhaps watching their little girl take steps of her own has been more important to them.
Though Mendes did admit in her exit interview that she did miss being a part of the reality TV show, Total Divas "I loved being on reality TV, I loved getting followed around by cameras and everyone staring at me… I gotta say, I just loved the attention [laughs]." (Source: WWE).
10 Candice Michelle
via way2enjoy.com
Candice Michelle had many moments in the ring that surely caught your attention, but nothing happened in her career that was perhaps as eye-catching as when she elected to appear in famous magazines. One person who was a huge fan of the decision was her husband, Ken Gee, who to his own credit is a very accomplished chiropractor. The two had met in 1999 and were married in 2005.
When Ken was asked about Candice posing in the issue, he replied "This has been a dream of hers since before we met and now that it has manifested it just shows all the hard work she has done. Candice has always wanted the cover because they typically only give it to celebrities. I guess she is a celebrity now, pretty weird, she's the same girl I fell in love with years ago, she just spends more money!" Thankfully landing photo deals with magazines would have allowed her to have more of that income to spend! That being said, thinking of the girl from the magazine cover and seeing how pregnant Candice looks in the above photo may definitely be jarring! Michelle and Ken have 3 children together, AkiAnne (born 2010), Ryumi (born 2012) and Aloha (born 2015). (Source: PlanetChriopractic).
9 Maria Kanellis
via wwe.fr
Maria Kanellis knows that people are always going to be critical of some of her talent when it comes to the wrestling industry. But Kanellis was definitely not having it when people started to get upset at her for getting pregnant, only shortly after making her reappearance in the WWE, "I signed a contract with WWE in March and had no plans of getting pregnant until my contract was up. So, yes I have only been signed for a few months this time but, I worked for WWE before. For 5 and a half years. I've also been working in wrestling for 13 years. AND I've been working since I was 12. I was a bus girl for my first job. Thus, I have been working for 23 years."
She didn't stop her rant there, going on to say, "I CAN get pregnant. I can even stay home while pregnant and have a job because WWE thinks I am important and this baby is important enough to do so. After 23 years of work, do I deserve maternity leave? I think so and so does WWE." Maria and her husband, Mike Bennett, welcomed their little girl Fredrica Bennett into the world this past April. It is their first child. (Source: WrestlingInc).
8 Brie Bella
via pinterest.com
There is no question that Brie Bella has put her body through a tremendous amount of punishment when she was making a name for herself in the wrestling industry. But when she was discussing how her pregnancy was making her feel with E! News, taking a tussle in the ring may definitely seem preferable "Pregnancy has kicked my butt I will admit. I was like, 'Whoa, I feel like I'm going through WrestleMania every day!'" Don't worry though, Brie did make sure to mention one part of her pregnancy that really stood out as being a positive "The nice thing is that right now I can take naps and not feel guilty."
Brie and Bryan Danielson welcomed their daughter, Birdie Joe Danielson, into the world back in May 2017.
It definitely won't be the last addition for the family, as when talking to E! News, Brie revealed "I have to make a comeback to wrestling before [having another child]. I miss that ring. But I told Bryan, I was just like, listen I want a second one…because he's like let's do it when Birdie's nine months! I was like, 'Whoa. You're not the one who gets pregnant and goes through the labor. So no.'" (Source: Upi, lifeandstylemag).
7 Beth Phoenix
Adam "Edge" Copeland's career in the WWE definitely didn't end in the way he had hoped, with injuries sending him permanently to the sidelines. But we're sure in hindsight he'd be pretty amazed at how his life has been going as of late. This includes being married to the former female superstar, Beth Phoenix since October 2016. The two have welcomed two little ones into the world, both daughters, one named Lyric who was born in December 2013 and Ruby who was born in May 2016. With parents like that, you definitely shouldn't be surprised if either (if not both) ladies take up an interest in the world of professional wrestling. Especially with so many amazing role models in the company that they can look up to for inspiration. Let's just hope they listen to Dad's advice to probably take a lot fewer dives from the top of the ladder.
Despite Copeland's busy career after wrestling, including a stint on the television series Vikings, he also always goes out of his way to be a strong Dad, saying in an interview with ESPN "I read to my kids every night. Not a missed night. If I'm in Ireland [filming 'Viking'], and with the time difference, I will film myself reading a book for that night and send it to Dropbox so Beth can still play it for them. So even while I'm sleeping or working, they still get our nightly routine of dad reading them to bed." (Source: ESPN).
6 Taryn Terrell
via pinimg.com
Taryn Terrell spent some time in the WWE, but she's one of the many examples from over the years of someone who couldn't quite get her footing. She did, however, manage to put together several years in TNA which means that even if you don't remember her time under Vince, that she may still be the source of some fond memories for you. Terrell and her husband, Joseph Dryden, whose career includes working as a stuntman, have 2 children together. Their first was a daughter who they named Emerson and was born in March 2014.
Though in an interview with Fanatix, she admitted that her child was not all good news, "Of course [there were some mixed feelings], every day I'm working towards being a better wrestler. I'm never satisfied; I'm always striving to be better. And then I have to take this time off. It's scary coming back – thinking is my body going to be able to handle this, am I mentally going to able to handle being back out on the road – because my mind is on my baby all the time, she's my world." Their 2nd child was a son, named Rhett, who Terrell and Dryden welcomed into the world back in December 2016. While Terrell won't be pulling off any more suplexes anytime soon, as she stepped away from the ring in October 2017, let's hope that she feels just as confident in entertaining her children. (Source: Diva-Dirt).
5 Mickie James
via twitter.com
There are many instances out there where starting a family ends up becoming a big strain on the relationship. But for Mickie James and her then-boyfriend, Nick Aldis, James giving birth to their son - Donovan - in September 2014 just pushed the two of them closer together. They got engaged not long after James gave birth and ended up being married in December 2015. When talking about becoming pregnant, James admitted that it wasn't exactly planned, "I was working on so many things at the time, I was already working on a bunch of different projects like a reality show. I was back in the [music] studio, and I wanted to talk to WWE about going back there or taking a position as a trainer. All these thoughts were there."
Thankfully though, James ended up having positive thoughts on the ordeal.
She said "Initially, we thought, 'What are we going to do?' It was a blessing in disguise. This is a great thing because I was at a point where I wasn't nailed down to anything at that moment to where I had to say, 'I have work to do and this and that.' It was the perfect opportunity for me." (Source: MiamiHerald).
4 Sara Lee
via instagram.com
Sara Lee's biggest claim to fame from the WWE came when she was announced as the winner of the 6th Tough Enough. But her time in the company came to an end in 2016 and not long after, Lee took to Instagram to share some new news with the wrestling world. That her, alongside her partner Wesley Blake - who still competes in NXT - were going to be welcoming a little one to the world, "Yes, I am indeed preggers and couldn't be more excited. Everything happens for a reason right?! Little baby in there has been kicking my butt for the last 13 weeks but I'm starting to get my energy back and about to head to the gym! Oh and yes @thewweblake is the daddy"
Perhaps it's not the most surprising that Lee wasn't going to allow her pregnancy to slow down her fitness routine!
She gave birth to their daughter, Piper, in May 2017. And while Dawn Marie's termination that happened around her pregnancy was a huge source of controversy, as we'll talk about, an insider for PWInsider reported that the reason Lee was let go from her contract was that she was unable to succeed in the ring. Seems like a pretty legitimate reason to be fired. (Source: SportsKeeda).
3 Trish Stratus
via pinterest.es
Trish Stratus has accomplished a tremendous amount during her time in the WWE and inspired many future performers. But one of the biggest accomplishments that Stratus has is the fact that she has managed to be in the same relationship since High School. While her and her husband, Ron, have surely had some difficulties in the past, it is seriously impressive to be with someone for that length of time. The two have two children together, a son named Maximus (born in 2013) and a daughter named Madison-Patricia (2017).
When Stratus was once asked by ParentsCanada as to what advice she would pass on to women out there who are expecting, she replied "I cannot stress enough the importance of eating well and exercising throughout your pregnancy. You have total control to create an environment that is nourishing for your baby. Be as educated as you can about the whole process." Trish's 2nd pregnancy definitely did not go as smoothly as her first one. Unlike with Maximus, Trish and Ron struggled with infertility issues; including Trish tragically suffering a miscarriage at one point. When talking about why she shared her story with ParentsCanada, Trish stated "At the time, I didn't hear about it, I didn't talk to people who were going through this. That's why I want to get out there and share my story and tell people you're not alone." (Source: ParentsCanada).
2 Dawn Marie
via bodybuilding.com
Dawn Marie definitely wasn't done expressing her passion for the wrestling industry when her contract was ended in 2006. But her then-recent pregnancy was definitely a factor and Marie ended up going after the company for wrongful termination as she was let go while she was still expecting. When discussing the matter with SLAM! Magazine, Marie explained "I love the business, I miss the business, and I never really finished. I had some weight issues after the pregnancy, but I'm trying really hard, I've been working out really hard, and hopefully (I'll be back) soon. Don't get me wrong, I'm loving being a a mom. But I'm getting antsy, and want to come back."
Marie may have had the passion, but sadly for her, nothing about the back half of her career lived up to her expectations and she never entered a WWE ring again. While the terms were not released, she ended up settling the lawsuit with the company back in 2007. You'd hope it was enough to at least make sure her daughter can live comfortably or go after the career path of her dreams. You can also imagine that Vince McMahon can consider himself pretty fortunate that a potential scandal of this magnitude didn't end up blowing up his company; as it may if the same thing happened in 2018. (Source: Slam, WrestlingInc).
1 Stephanie McMahon
Stephanie McMahon definitely seems to be in a great mood in the above photo. Which means that while she may have been more than happy to smile for, logically, Paul Levesque (aka Triple H), there is probably a pretty good chance she never expected this photo to see the internet. Judging by the medical supplies that are behind her, it definitely looks like Stephanie was perhaps only moments away from giving birth. Or at the very least was going in for one of her last check-ups because she is definitely heavily pregnant. Good thing she'd have Triple H around to help with all of the potential heavy lifting!
The family has 3 children together, all daughters. Their first was born in 2006, another in 2008 and their 3rd in 2010. They named them Aurora, Murphy, and Vaughn. When talking about if they were interested in the sport, McMahon stated "They do watch it, my oldest and youngest in particular. My oldest [then aged 10] would love to be the women's champion and my youngest [then aged 6] wants to marry John Cena – so she needs to hurry up because he and Nikki Bella are now engaged. It is conflict resolution and the only difference is ours are settled in the ring in some great live action." Which also means Vaughn may have been happy at the news that they split! (Source: SportsKeeda).
Tommaso Ciampa Says He's Going Back To NXT When He's Cleared To Wrestle Again
Ember Moon Accuses Dana Brooke Of Stealing From Her Moveset
The 10 Most Violent And Bloody Matches In ECW History
10 Best ECW Factions Of All Time, Ranked
10 TNA Wrestlers That WWE Did Not Want To Succeed
From The Undertaker To Sting: 5 Legends Who Should Have One More Match (& 5 Who Should Never Wrestle Again)
Truly Weds: 5 Wrestling Couples That Were Really Married (And 5 That Weren't)

redpajama_set_name: RedPajamaCommonCrawl


It is with our deepest regret and sorrow that we announce the sudden passing of Kamal Haddad of 108 Mile Ranch, B.C. We invite you to gather with family and friends for a time of remembrance at the 108 Mile Resort & Conference Centre on Thursday, Dec. 21, 2006 at 1:00 p.m. In lieu of flowers, please make donations to the 100 Mile & District SPCA.
100 Mile Funeral Service Ltd. entrusted with the arrangments.


text:
C2hau (talk | contribs)
(→Class 2 - Thursday, May 9)
Conversion script (talk | contribs)
m (Conversion script moved page Stat340s13 to stat340s13: Converting page titles to lowercase)
=== Final ===
Saturday August 10,2013 from 7:30pm-10:00pm
=== TA(s): ===
Another algorithm for generating pseudo random numbers is the multiply with carry method. Its simplest form is similar to the linear congruential generator. They differs in that the parameter b changes in the MWC algorithm. It is as follows:

1.) xk+1 = axk + bk mod m

2.) bk+1 = floor((axk + bk)/m)

[http://www.javamex.com/tutorials/random_numbers/multiply_with_carry.shtml Source]
=== Inverse Transform Method ===

>>hist(u) #will generate a fairly uniform diagram
>>hist(u) # this will generate a fairly uniform diagram
[[File:ITM_example_hist(u).jpg|300px]]
Sol:
Let <math>y=x^5</math>, solve for x: <math>x=y^\frac {1}{5}</math>. Therefore, <math>F^{-1} (x) = x^\frac {1}{5}</math>

Hence, to obtain a value of x from F(x), we first set u as an uniform distribution, then obtain the inverse function of F(x), and set
Hence, to obtain a value of x from F(x), we first set 'u' as an uniform distribution, then obtain the inverse function of F(x), and set
<math>x= u^\frac{1}{5}</math>

Note that after generating a random U, the value of X can be determined by finding the interval <math>[F(x_{j-1}),F(x_{j})]</math> in which U lies.

Generate a discrete r.v.x that has pmf:

P(X=xi)=Pi, x0<x1<x2<...

1. Draw U~U(0,1);

2. If F(x(i-1))<U<F(xi), x=xi.

== Class 4 - Thursday, May 16 ==
*When we want to find target distribution, denoted as <math>f(x)</math>, we need to first find a proposal distribution <math>g(x)</math> that is easy to sample from.

'''Goals'''

*When we want to find target distribution <math>f(x)</math>, we need to first find a proposal distribution <math>g(x)</math> that is easy to sample from.

*Relationship between the proposal distribution and target distribution is: <math> c \cdot g(x) \geq f(x) </math>, where c is constant. This means that the area of f(x) is under the area of <math> c \cdot g(x)</math>.

*Chance of acceptance is less if the distance between <math>f(x)</math> and <math> c \cdot g(x)</math> is big, and vice-versa, we use <math> c </math> to keep <math> \frac {f(x)}{c \cdot g(x)} </math> below 1 (so <math>f(x) \leq c \cdot g(x)</math>). Therefore, we must find the constant <math> C </math> to achieve this.

*In other words, <math>C</math> is chosen to make sure <math> c \cdot g(x) \geq f(x) </math>. However, it will not make sense if <math>C</math> is simply chosen to be arbitrarily large. We need to choose <math>C</math> such that <math>c \cdot g(x)</math> fits <math>f(x)</math> as tightly as possible. This means that we must find the minimum c such that the area of f(x) is under the area of c*g(x).

*The constant c cannot be a negative number.

'''How to find C''':

<math>\begin{align}
&c \cdot g(x) \geq f(x)\\
&c= \max \left(\frac{f(x)}{g(x)}\right)
\end{align}</math>

If <math>f</math> and <math> g </math> are continuous, we can find the extremum by taking the derivative and solve for <math>x_0</math> such that:

<math> 0=\frac{d}{dx}\frac{f(x)}{g(x)}|_{x=x_0}</math>

Thus <math> c = \frac{f(x_0)}{g(x_0)} </math>

*The logic behind this:
Note: This procedure is called the Acceptance-Rejection Method.

The Acceptance-Rejection method involves finding a distribution that we know how to sample from, g(x), and multiplying g(x) by a constant c so that <math>c \cdot g(x)</math> is always greater than or equal to f(x). Mathematically, we want <math> c \cdot g(x) \geq f(x) </math>.
And it means, c has to be greater or equal to <math>\frac{f(x)}{g(x)}</math>. So the smallest possible c that satisfies the condition is the maximum value of <math>\frac{f(x)}{g(x)}</math>
. If c is too large, the chance of acceptance of generated values will be small, thereby losing efficiency of the algorithm. Therefore, it is best to get the smallest possible c such that <math> c g(x) \geq f(x)</math>.

'''The Acceptance-Rejection method''' involves finding a distribution that we know how to sample from, g(x), and multiplying g(x) by a constant c so that <math>c \cdot g(x)</math> is always greater than or equal to f(x). Mathematically, we want <math> c \cdot g(x) \geq f(x) </math>.
And it means, c has to be greater or equal to <math>\frac{f(x)}{g(x)}</math>. So the smallest possible c that satisfies the condition is the maximum value of <math>\frac{f(x)}{g(x)}</math>
.
But in case of c being too large, the chance of acceptance of generated values will be small, thereby losing efficiency of the algorithm. Therefore, it is best to get the smallest possible c such that <math> c g(x) \geq f(x)</math>.

'''Important points:'''

*For this method to be efficient, the constant c must be selected so that the rejection rate is low. (The efficiency for this method is <math>\left ( \frac{1}{c} \right )</math>)

*It is easy to show that the expected number of trials for an acceptance is <math> \frac{Total Number of Trials} {C} </math>.

*recall the acceptance rate is 1/c. (Not rejection rate)
*recall the '''acceptance rate is 1/c'''. (Not rejection rate)
:Let <math>X</math> be the number of trials for an acceptance, <math> X \sim~ Geo(\frac{1}{c})</math>

:<math>\mathbb{E}[X] = \frac{1}{\frac{1}{c}} = c </math>
*The number of trials needed to generate a sample size of <math>N</math> follows a negative binomial distribution. The expected number of trials needed is then <math>cN</math>.

*So far, the only distribution we know how to sample from is the '''UNIFORM''' distribution.

'''Procedure''':

1. Choose <math>g(x)</math> (simple density function that we know how to sample, i.e. Uniform so far)

The easiest case is <math>U~ \sim~ Unif [0,1] </math>. However, in other cases we need to generate UNIF(a,b). We may need to perform a linear transformation on the <math>U~ \sim~ Unif [0,1] </math> variable.

#If <math>U \leq \frac{f(Y)}{c \cdot g(Y)}</math> then X=Y; else return to step 1 (This is not the way to find C. This is the general procedure.)

Example: Generate a random variable from the pdf

Example:

Generate a random variable from the pdf

<math> f(x) =
\begin{cases}
[[File:Beta(2,1)_example.jpg|750x750px]]
Note: g follows uniform distribution, it only covers half of the graph which runs from 0 to 1 on y-axis. Thus we need to multiply by c to ensure that <math>c\cdot g</math> can cover entire f(x) area. In this case, c=2, so that makes g run from 0 to 2 on y-axis which covers f(x).
'''Note:''' g follows uniform distribution, it only covers half of the graph which runs from 0 to 1 on y-axis. Thus we need to multiply by c to ensure that <math>c\cdot g</math> can cover entire f(x) area. In this case, c=2, so that makes g run from 0 to 2 on y-axis which covers f(x).
'''Comment:'''

From the picture above, we could observe that the area under f(x)=2x is a half of the area under the pdf of UNIF(0,1). This is why in order to sample 1000 points of f(x), we need to sample approximately 2000 points in UNIF(0,1).
And in general, if we want to sample n points from a distritubion with pdf f(x), we need to scan approximately <math>n\cdot c</math> points from the proposal distribution (g(x)) in total.

'''Note:''' In the above example, we sample 2 numbers. If second number (u) is less than or equal to first number (y), then accept x=y, if not then start all over.
Matlab Code
To obtain a better proposing function <math>\begin{align}g(x)\end{align}</math>, we can first assume a new <math>\begin{align}q(x)\end{align}</math> and then solve for the normalizing constant by integrating.

In the previous example, we first assume <math>\begin{align}q(x) = 3x\end{align}</math>. To find the normalizing constant, we need to solve <math>k *\sum 3x = 1</math> which gives us k = 2/3. So,<math>\begin{align}g(x) = k*q(x) = 2x\end{align}</math>.
*Source: http://www.cs.bgu.ac.il/~mps042/acceptance.htm*
'''Possible Limitations'''
Note: In this case, <math>\begin{align}f(x)/g(x)\end{align}</math> is extremely difficult to differentiate so we were required to test points. If the function is very easy to differentiate, we can calculate the max as if it were a continuous function then check the two surrounding points for which is the highest discrete value.
* Source: http://www.math.wsu.edu/faculty/genz/416/lect/l04-46.pdf*
*'''Example 4''' (Hypergeometric & Binomial)

The CDF of the Gamma distribution <math>Gamma(t,\lambda)</math> is(t denotes the shape, <math>\lambda</math> denotes the scale:

<math> F(x) = \int_0^{x} \frac{e^{-y}y^{t-1}}{(t-1)!} \mathrm{d}y, \; \forall x \in (0,+\infty)</math>, where <math>t \in \N^+ \text{ and } \lambda \in (0,+\infty)</math>.

Note that the CDF of the Gamma distribution does not have a closed form.
The gamma distribution is often used to model waiting times between a certain number of events. It can also be expressed as the sum of infinitely many independent and identically distributed exponential distributions. This distribution has two parameters: the number of exponential terms n, and the rate parameter <math>\lambda</math>. In this distribution there is the Gamma function, <math>\Gamma </math> which has some very useful properties. "Source: STAT 340 Spring 2010 Course Notes"

:<math>f(x) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} x^2}</math>
*Warning : the General Normal distribution is
*Warning : the General Normal distribution is:
:<math>f(y) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} y^2}</math>
:<math>f(x,y) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} x^2} * \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} y^2}=\frac{1}{2\pi}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} (x^2+y^2)} </math>
 - Since for independent distributions, their joint probability function is the multiplication of two independent probability functions
:<math>f(x,y) = \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} x^2} * \frac{1}{\sqrt{2\pi}}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} y^2}=\frac{1}{2\pi}\, e^{- \frac{\scriptscriptstyle 1}{\scriptscriptstyle 2} (x^2+y^2)} </math>
 - Since for independent distributions, their joint probability function is the multiplication of two independent probability functions. It can also be shown using 1-1 transformation that the joint distribution of R and θ is given by, 1-1 transformation:

It can also be shown using 1-1 transformation that the joint distribution of R and θ is given by,
1-1 transformation:

Let <math>d=R^2</math>

'''Let <math>d=R^2</math>'''

<math>x= \sqrt {d}\cos \theta </math>
<math>y= \sqrt {d}\sin \theta </math>
<math>\begin{align} f(\theta) =\frac{1}{2\pi}\ \end{align}</math>
To sample from the normal distribution, we can generate a pair of independent standard normal X and Y by:

1) Generating their polar coordinates

2) Transforming back to rectangular (Cartesian) coordinates.

Step 1: Generate <math>u_{1}</math> ~<math>Unif(0,1)</math>
'''Alternative Method of Generating Standard Normal Random Variables'''

Step 2: Generate <math>Y_{1}</math> ~<math>Exp(1)</math>,<math>Y_{2}</math>~<math>Exp(2)</math>
Step 3: If <math>Y_{2} \geq(Y_{1}-1)^2/2</math>,set <math>V=Y1</math>,otherwise,go to step 1
Step 1: Generate <math>u_{1}</math> ~<math>Unif(0,1)</math>

Step 4: If <math>u_{1} \leq 1/2</math>,then <math>X=-V</math>
Step 2: Generate <math>Y_{1}</math> ~<math>Exp(1)</math>,<math>Y_{2}</math>~<math>Exp(2)</math>

Step 3: If <math>Y_{2} \geq(Y_{1}-1)^2/2</math>,set <math>V=Y1</math>,otherwise,go to step 1

Step 4: If <math>u_{1} \leq 1/2</math>,then <math>X=-V</math>

===Expectation of a Standard Normal distribution===

The expectation of a standard normal distribution is 0

==== Expectation of a Standard Normal distribution ====
'''Proof:'''

:Below is the proof:
:<math>\operatorname{E}[X]= \;\int_{-\infty}^{\infty} x \frac{1}{\sqrt{2\pi}} e^{-x^2/2} \, dx.</math>
:<math>= - \left[\phi(x)\right]_{-\infty}^{\infty}</math>
:<math>= 0</math>

More intuitively, because x is an odd function (f(x)+f(-x)=0). Taking integral of x will give <math>x^2/2 </math> which is an even function (f(x)=f(-x)). Note that this is in relation to the symmetrical properties of the standard normal distribution. If support is from negative infinity to infinity, then the integral will return 0.

* '''Procedure (Box-Muller Transformation Method):'''

'''Note,''' more intuitively, because x is an odd function (f(x)+f(-x)=0). Taking integral of x will give <math>x^2/2 </math> which is an even function (f(x)=f(-x)). This is in relation to the symmetrical properties of the standard normal distribution. If support is from negative infinity to infinity, then the integral will return 0.

'''Procedure (Box-Muller Transformation Method):'''

Pseudorandom approaches to generating normal random variables used to be limited. Inefficient methods such as inverse Gaussian function, sum of uniform random variables, and acceptance-rejection were used. In 1958, a new method was proposed by George Box and Mervin Muller of Princeton University. This new technique was easy to use and also had the accuracy to the inverse transform sampling method that it grew more valuable as computers became more computationally astute.

The Box-Muller method takes a sample from a bivariate independent standard normal distribution, each component of which is thus a univariate standard normal. The algorithm is based on the following two properties of the bivariate independent standard normal distribution:

if <math>Z = (Z_{1}, Z_{2}</math>) has this distribution, then

1.<math>R^2=Z_{1}^2+Z_{2}^2</math> is exponentially distributed with mean 2, i.e.

<math>P(R^2 \leq x) = 1-e^{-x/2}</math>.

2.Given <math>R^2</math>, the point <math>(Z_{1},Z_{2}</math>) is uniformly distributed on the circle of radius R centered at the origin.

We can use these properties to build the algorithm:

1) Generate random number <math> \begin{align} U_1,U_2 \sim~ \mathrm{Unif}(0, 1) \end{align} </math>

Note: In steps 2 and 3, we are using a similar technique as that used in the inverse transform method.

'''Note:''' In steps 2 and 3, we are using a similar technique as that used in the inverse transform method.

The Box-Muller Transformation Method generates a pair of independent Standard Normal distributions, X and Y (Using the transformation of polar coordinates).

If you want to generate a number of independent standard normal distributed numbers (more than two), you can run the Box-Muller method several times.

For example:

* '''Code'''

'''Matlab Code'''

Comment: Box-Muller transformations are not computationally efficient. The reason for this is the need to compute sine and cosine functions. A way to get around this time-consuming difficulty is by an indirect computation of the sine and cosine of a random angle (as opposed to a direct computation which generates U and then computes the sine and cosine of 2πU.

Comment:

Box-Muller transformations are not computationally efficient. The reason for this is the need to compute sine and cosine functions. A way to get around this time-consuming difficulty is by an indirect computation of the sine and cosine of a random angle (as opposed to a direct computation which generates U and then computes the sine and cosine of 2πU.

'''Alternative Methods of generating normal distribution'''

1. Even though we cannot use inverse transform method, we can approximate this inverse using different functions.One method would be '''rational approximation'''.

2.'''Central limit theorem''' : If we sum 12 independent U(0,1) distribution and subtract 6 (which is E(ui)*12)we will approximately get a standard normal distribution.

=== Proof of Box Muller Transformation ===
'''Definition:'''

A transformation which transforms from a '''two-dimensional continuous uniform''' distribution to a '''two-dimensional bivariate normal''' distribution (or complex normal distribution).
Step1: Generate U~ U(0, 1)

Step2: set <math>y=\, {-\frac {1}{{\lambda_1 +\lambda_2}}} ln(u)</math>

Step2: set <math>y=\, {-\frac {1}{{\lambda_1 +\lambda_2}}} ln(1-u)</math>

or set <math>y=\, {-\frac {1} {{\lambda_1 +\lambda_2}}} ln(u)</math>

Since it is a uniform distribution, therefore after generate a lot of times 1-u and u are the same.
* '''Matlab Code'''

===Poisson Process===
A Poisson Process is a stochastic approach to count number of events in a certain time period. <s>Strike-through text</s>
A discrete stochastic variable ''X'' is said to have a Poisson distribution with parameter ''λ'' > 0 if
:<math>\!f(n)= \frac{\lambda^n e^{-\lambda}}{n!} \qquad n= 0,1,2,3,4,5,\ldots,</math>.
'''Generate a Poisson Process'''

<math>U_{n} \sim~ U(0,1)</math>

<math>T_n-T_{n-1}=-\frac {1}{\lambda} log(U_n)</math>

1. set <math>T_{0}=0</math> and n=1

<math>\pi</math> is stationary distribution of the chain if <math>\pi</math>P = <math>\pi</math>
<math>\pi</math> is stationary distribution of the chain if <math>\pi</math>P = <math>\pi</math> In other words, a stationary distribution is when the markov process that have equal probability of moving to other states as its previous move.
where <math>\pi</math> is a probability vector <math>\pi</math>=(<math>\pi</math>i | <math>i \in X</math>) such that all the entries are nonnegative and sum to 1. It is the eigenvector in this case.
<math> \pi_0 = \frac{4}{19} </math>

<math> \pi = [\frac{4}{19}, \frac{15}{19}] </math>

<math> \pi </math> is the long run distribution
<math> \pi </math> is the long run distribution, and this is also a limiting distribution.
We can use the stationary distribution to compute the expected waiting time to return to state 'a'

state 'a' given that we start at state 'a' is 19/4.

definition of limiting distribution: when the stationary distribution is convergent, it is a limiting distribution.

remark:satisfied balance of <math>\pi_i P_{ij} = P_{ji} \pi_j</math>, so there is other way to calculate the step probability.
<math>P_i= (1-d) + d\cdot \sum_j \frac {L_{ji}P_j}{c_j}</math>
pi is the rank of a new created page(that no one knows about) is 0 since <math>L_ij</math> is 0

where 0 < d < 1 is constant (in original page rank algorithm d = 0.8), and <math>L_{ij}</math> is 1 if j has link to i, 0 otherwise.
4) finally, we take a linear combination of the page rank obtained from above and a constant 1. This ensures that every page has a rank greater than zero.

5) d is the damping factor. It represents the probability a user, at any page, will continue clicking to another page.

Note that this is a system of N equations with N unknowns.

It is, in general, very difficult to simulate the value of a random vector X whose component random variables are dependent. We will present a powerful approach for generating a vector whose distribution is approximately that of X. This approach, called the Markov Chain Monte Carlo Methods, has the added significance of only requiring that the mass(or density) function of X be specified up to a multiplicative constant, and this, we will see, is of great importance in applications.
(referenced by Sheldon M.Ross,Simulation)
The basic idea used here is to generate a Markov Chain whose stationary dsitribution is the same as the target distribution.
The basic idea used here is to generate a Markov Chain whose stationary distribution is the same as the target distribution.
====Definition:====
<math>q(y|x)</math> is used instead of <math>qi,j</math>. In continuous case, we use these notation which means given state x, what's the probability of y.

Note that the Metropolis-Hasting algorithm possess some advantageous properties. One of which is that this algorithm "can be used when \pi(x) is known up to the constant of proportionality". The second is that in this algorithm, "we do not require the conditional distribution, which, in contrast, is required for the Gibbs sampler. "
Source:https://www.msu.edu/~blackj/Scan_2003_02_12/Chapter_11_Markov_Chain_Monte_Carlo_Methods.pdf
'''Differences between the discrete and continuous case of the Markov Chain''':

  Motivation: Simulated Annealing and the Travelling Salesman Problem

The Travelling Salesman Problem asks:

Given n numbers of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the original city?
Given n numbers of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the original city? By letting two permutations if one results from an interchange of two of the coordinates of the other, we can use simulated annealing to approximate the best path.

[[File:Salesman n5.png|350px]]
(http://en.wikipedia.org/wiki/Gibbs_sampling)

Gibbs sampling is a general method for probabilistic inference which is often used when dealing with incomplete information. However, generality comes at some computational cost, and for many applications including those involving missing information, there are often alternative methods that have been shown to be more efficient in practice. Suppose we want to sample from a joint distribution <math>p(x_1,...,x_k)</math> (i.e. a posterior distribution). If we knew the full conditional distributions for each parameter (i.e. <math>p(x_i|x_1,x_2,...,x_{i-1},x_{i+1},...,x_k)</math>), we can use the Gibbs sampler to sample from these conditional distributions.

The Gibbs sampling method was originally developed by Geman and Geman [1984]. It was later brought into mainstream statistics by Gelfand and Smith [1990] and Gelfand, et al. [1990]

Source: https://www.msu.edu/~blackj/Scan_2003_02_12/Chapter_11_Markov_Chain_Monte_Carlo_Methods.pdf

Gibbs sampling is a general method for probabilistic inference which is often used when dealing with incomplete information. However, generality comes at some computational cost, and for many applications including those involving missing information, there are often alternative methods that have been proven to be more efficient in practice. For example, say we want to sample from a joint distribution <math>p(x_1,...,x_k)</math> (i.e. a posterior distribution). If we knew the full conditional distributions for each parameter (i.e. <math>p(x_i|x_1,x_2,...,x_{i-1},x_{i+1},...,x_k)</math>), we can use the Gibbs sampler to sample from these conditional distributions.

When utilizing the Gibbs sampler, the candidate state is always accepted as the nect state of the chain.(from text book)

When utilizing the Gibbs sampler, the candidate state is always accepted as the next state of the chain.(from text book)

*Another Markov Chain Monte Carlo (MCMC) method (first MCMC method introduced in this course is the MH Algorithm)

'''Difference between Gibbs Sampling & MH'''

Gibbs Sampling generates new value based on the conditional distribution of other components.

Gibbs Sampling generates new value based on the conditional distribution of other components (unlike MH, which does not require conditional distribution).

eg. We are given the following about <math> f(x_1,x_2) , f(x_1|x_2),f(x_2|x_1) </math>

1. let <math>x^*_1 \sim f(x_1|x_2)</math>

2. <math>x^*_2 \sim f(x^*_1|x_2)</math>

2. <math>x^*_2 \sim f(x_2|x^*_1)</math>

3. substitute <math>x^*_2</math> back into first step and repeat the process.

Also, for Gibbs sampling, we will "always accept a candidate point", unlike MH

'''Gibbs Sampling as a special form of the Metropolis Hastings algorithm'''

The Gibbs Sampler is simply a case of the Metropolis Hastings algorithm

here, the proposal distribution is <math>q(Y|X)=f(X^j|X^*_i, i\neq j)=\frac{f(Y)}{f(X_i, i\neq j)}</math> for <math>X=(X_1,...,X_n)</math>,

which is simply the conditional distribution of each element conditional on all the other elements in the vector.

similarly <math>q(X|Y)=f(X|Y^*_i, i\neq j)=\frac{f(X)}{f(Y_i, i\neq j)}</math>

notice that <math>(Y_i, i\neq j)</math> and <math>(X_i, i\neq j)</math> are identically distributed.

the distribution we wish to simulate from is <math>p(X) = f(X) </math>
also, <math>p(Y) = f(Y) </math>
Hence, the acceptance ratio in the Metropolis-Hastings algorithm is:

<math>r(x,y) = min\left\{\frac{f(x)}{f(y)}\frac{q(y|x)}{q(x|y)},1\right\} = min\left\{\frac{f(x)}{f(y)}\frac{f(y)}{f(x)},1\right\} = 1 </math>

so the new point will always be accepted, and no points are rejected and the Gibbs Sampler is an efficient algorithm in that aspect.

Advantages <ref>
Let's suppose that we are interested in sampling from the posterior p(x|y), where x is a vector of three parameters, x1, x2, x3.

The steps to a Gibbs Sampler are:

1. Pick a vector of starting value x(0).

1. Pick a vector of starting value x(0). Any x(0) will converge eventually, but it can be chosen to take fewer iterations

2. Start with any x(order does not matter, but I will start with x1 for convenience). Draw a value x1(1)from the full conditional p(x1|x2(0),x3(0),y)

3. Draw a value x2(1) from the full conditional p(x2|x1(1),x3(0),y). Note that we must use the updated value of x1(1).

'''Example:''' Face recognition

X is a greyscale image of the perso and Y is the person.

X is a greyscale image of the person and Y is the person.

Here,We have a 100 x 100 grid where each cell is a number from 0 to 255 representing the darkness of the cell (from white to black).

Let x be a vector of length 100*100=10,000 and y be a vector with each element being a picture of a person's face.

[[Frequentist approach]]

*A frequentist would say X is a random variable and Y is not, so they would use Pr{x|y} (given that y is Tom, how likely is it that x is an image of Tom?).
<math>\displaystyle P(X|Y)</math>, y is person and x is how likely the picture is of this person. Here, y is known.

'''Definition'''

*Variance reduction is a procedure used to increase the precision of the estimates that can be obtained for a given number of iterations. Every output random variable from the simulation is associated with a variance which limits the precision of the simulation results.

*In order to make a simulation statistically efficient,(i.e. to obtain a greater precision and smaller confidence intervals for the output random variable of interest),variance reduction techniques can be used. The main ones are: Common random numbers, antithetic variates, control variates, importance sampling and stratified sampling), We will only be learning one of the methods - importance sampling. http://en.wikipedia.org/wiki/Variance_reduction
*In order to make a simulation statistically efficient,(i.e. to obtain a greater precision and smaller confidence intervals for the output random variable of interest),variance reduction techniques can be used. The main ones are: Common random numbers, antithetic variates, control variates, importance sampling and stratified sampling), We will only be learning one of the methods - importance sampling. Importance sampling is used to generate more statistically significant points rather than generating those points that do not have any value, such as generating in the middle of the bell curve rather than at the tail end of the bell curve. http://en.wikipedia.org/wiki/Variance_reduction
*
It can be seen that the integral <math>\displaystyle\int \frac{h(x)f(x)}{g(x)}g(x)\,dx = \int \frac{f(x)}{g(x)}h(x) g(x)\,dx</math> is just <math> \displaystyle E_g(h(x)) \rightarrow</math>the expectation of h(x) with respect to g(x), where <math>\displaystyle \frac{f(x)}{g(x)} </math> is a weight <math>\displaystyle\beta(x)</math>. In the case where <math>\displaystyle f > g</math>, a greater weight for <math>\displaystyle\beta(x)</math> will be assigned.
<ref>
http://wikicoursenote.com/wiki/Stat341#Importance_Sampling_2
*Variance reduction uses the fact that the variance of a finite integral is zero.

We wish to use simulation for this algorithm. We can utilize Monte Carlo Integration framework from previous classes.
We would like to use simulation for this algorithm. We can use Monte Carlo Integration framework from previous classes.
<math>E_f [h(x)] = \int h(x)f(x) dx</math>. The motivation is that a lot of integrals need to be calculated.

'''Some addition knowledge:'''

Common Random Numbers: The common random numbers variance reduction technique is a popular and useful variance reduction technique which applies when we are comparing two or more alternative configurations (of a system) instead of investigating a single configuration.

Common Random Numbers: The common random numbers variance reduction technique is a popular and useful variance reduction technique which applies when we are comparing at least two alternative configurations (of a system) instead of investigating a single configuration.

'''Case 1 Basic Monte Carlo Integration'''

'''Idea:'''Evaluating an integral means calculating the area under the desired curve f(x).

Consider <math>I = \int_0 ^1 e^x dx</math>

The exact answer is e - 1.
The exact answer is (e^1 - e^0) = 2.718281828 - 1 = 1.718281828
Comparing to the simulation, the matlab code is as follows:
If <math>x~\sim U(0,1)</math> and hence <math>\,f(x)=1</math>, then we have the basis of other variance reduction techniques. Now we consider what happens if X is not uniformly distributed.
In the control variate case, we change the formula b adding and subtracting a known function h(x): basically, by adding zero to the integral, keeping it unbiased and allowing us to have an easier time of solving it. In importance sampling, we will instead multiply by 1. The known function in this case will be g(x), which is selected under a few assumptions.
There are cases where another distribution gives a better fit to integral to approximate, and results in a more accurate estimate; importance sampling is useful here.
==Class 24, Thursday, July 25, 2013==
===Importance Sampling===
Importance Sampling is the most fundamental variance reduction technique, and often leads to a dramatic variance reduction.

Importance Sampling is the most fundamental variance reduction technique and usually leads to a dramatic variance reduction.

Importance sampling involves choosing a sampling distribution that favor important samples*.(Simulation and the Monte Carlo Method, Reuven Y. Rubinstein)

Importance sampling involves choosing a sampling distribution that favour important samples*.(Simulation and the Monte Carlo Method, Reuven Y. Rubinstein)

* Here "favor important samples" means to encourage the occurrence of the desired event or part of the desired event. For instance, if the event of interest is rare (probability of occurrence is close to zero), we "favor important samples" by choosing a sampling distribution such that the event has higher probability of occurrence.
* Here "favour important samples" implies encouraging the occurrence of the desired event or part of the desired event. For instance, if the event of interest is rare (probability of occurrence is close to zero), we "favour important samples" by choosing a sampling distribution such that the event has higher probability of occurrence.
Definition of importance sampling from Wikipedia:

note: in summary, a good importance sampling function g(x) should satisfies:

1. g(x) > 0 whenver f(x)not equal to 0

1. g(x) > 0 whenever f(x)not equal to 0

2. g(x) should be equal or close to the absolute value of f(x)

3. easy to simulate values from g(x)

<math>\widehat{I}=\frac{1}{n}\sum_{i=1}^{n}w(x)</math> and <math>g^{*}(x) = \frac{|h(x)|f(x)}{\int |h(x)|f(x)dx}</math>, where <math> h(x)>=0 </math> for all x

Higher values of n correspond to values of <math>\widehat{I}</math> closer to <math>{I}</math>, which approaches <math>\widehat{I}</math> as n approaches infinity.
'''Note:''' g(x) should be chosen carefully. It should be easy to sample from. Also, since this method is for minimizing the variance, g(x) should be chosen in a manner such that the variance is minimized. g*(x) is the distribution that minimizes the variance.
===Final Exam Review===
'''Summary of Final Exam Topics:'''
''Pre-Midterm:''
•	Multiplicative Congruential Algorithm
•	Inverse Transform Method
•	Acceptance Rejection Method
•	Multivariate Random Variable Generation
•	Vector Acceptance Rejection Method
''Post-Midterm:''
•	Poisson Process
•	Markov Chains (MC)
•	Page Rank (MC Application)
•	Markov Chain Monte Carlo (MCMC)
•	Metropolis-Hasting Algorithm (MCMC Application)
•	Simulated Annealing (MCMC Application)
•	Gibbs Sampling (Metropolis-Hasting adaptation)
•	Monte Carlo Integration
•	Importance Sampling
Only review of material not covered on the midterm, the final will be cumulative

For review of material covered on the midterm refer to class 12 - June 13th.
*The two most important stochastic processes we looked at in this term are Poission Process and Markov Chain.

'''Poisson Process''' (useful for counting number of arrivals):

===Poisson Process=== (useful for counting number of arrivals):

- two assumptions:

# the number of arrivals in non-overlapping intervals are independent

*Marginal distribution:<math>\mu_1~ = \mu_0P</math>

In general, <math>\mu_n~ = \mu_0P^n</math>

where <math> \mu_0</math> is initial dust.

*'''Stationary distribution''': <math>\pi</math> = <math>\pi</math> P

There are three conditions to calculate Stationary Distribution

1. <math>\mu_1~ = \mu_0P</math>

2. sum of <math>\pi</math>= 1

3. <math>\pi</math> is greater than 0

*'''Limiting distribution''':
<math>\lim_{n\to \infty} P^n= \left[ {\begin{array}{ccc}
3.) set k to k + 1 and go to step 1 Source
>>hist(u)       # this will generate a fairly uniform diagram
Hence, to obtain a value of x from F(x), we first set 'u' as an uniform distribution, then obtain the inverse function of F(x), and set [math]x= u^\frac{1}{5}[/math]
In summary: Generate a discrete r.v.x that has pmf:
P(X=xi)=Pi,    x0<x1<x2<...
1. Draw U~U(0,1);
2. If F(x(i-1))<U<F(xi), x=xi.
Source: http://www.cs.bgu.ac.il/~mps042/acceptance.htm*
Source: http://www.math.wsu.edu/faculty/genz/416/lect/l04-46.pdf*
Step2: set [math]y=\, {-\frac {1}{{\lambda_1 +\lambda_2}}} ln(1-u)[/math]
or set [math]y=\, {-\frac {1} {{\lambda_1 +\lambda_2}}} ln(u)[/math]
A Poisson Process is a stochastic approach to count number of events in a certain time period. Strike-through text A discrete stochastic variable X is said to have a Poisson distribution with parameter λ > 0 if
[math]\pi[/math] is stationary distribution of the chain if [math]\pi[/math]P = [math]\pi[/math] In other words, a stationary distribution is when the markov process that have equal probability of moving to other states as its previous move.
[math] \pi [/math] is the long run distribution, and this is also a limiting distribution.
definition of limiting distribution: when the stationary distribution is convergent, it is a limiting distribution.
It is, in general, very difficult to simulate the value of a random vector X whose component random variables are dependent. We will present a powerful approach for generating a vector whose distribution is approximately that of X. This approach, called the Markov Chain Monte Carlo Methods, has the added significance of only requiring that the mass(or density) function of X be specified up to a multiplicative constant, and this, we will see, is of great importance in applications. (referenced by Sheldon M.Ross,Simulation) The basic idea used here is to generate a Markov Chain whose stationary distribution is the same as the target distribution.
The Gibbs sampling method was originally developed by Geman and Geman [1984]. It was later brought into mainstream statistics by Gelfand and Smith [1990] and Gelfand, et al. [1990]
Source: https://www.msu.edu/~blackj/Scan_2003_02_12/Chapter_11_Markov_Chain_Monte_Carlo_Methods.pdf
Gibbs sampling is a general method for probabilistic inference which is often used when dealing with incomplete information. However, generality comes at some computational cost, and for many applications including those involving missing information, there are often alternative methods that have been proven to be more efficient in practice. For example, say we want to sample from a joint distribution [math]p(x_1,...,x_k)[/math] (i.e. a posterior distribution). If we knew the full conditional distributions for each parameter (i.e. [math]p(x_i|x_1,x_2,...,x_{i-1},x_{i+1},...,x_k)[/math]), we can use the Gibbs sampler to sample from these conditional distributions.
When utilizing the Gibbs sampler, the candidate state is always accepted as the next state of the chain.(from text book)
Gibbs Sampling generates new value based on the conditional distribution of other components (unlike MH, which does not require conditional distribution).
2. [math]x^*_2 \sim f(x_2|x^*_1)[/math]
Also, for Gibbs sampling, we will "always accept a candidate point", unlike MH
Gibbs Sampling as a special form of the Metropolis Hastings algorithm
The Gibbs Sampler is simply a case of the Metropolis Hastings algorithm
here, the proposal distribution is [math]q(Y|X)=f(X^j|X^*_i, i\neq j)=\frac{f(Y)}{f(X_i, i\neq j)}[/math] for [math]X=(X_1,...,X_n)[/math],
which is simply the conditional distribution of each element conditional on all the other elements in the vector.
similarly [math]q(X|Y)=f(X|Y^*_i, i\neq j)=\frac{f(X)}{f(Y_i, i\neq j)}[/math]
notice that [math](Y_i, i\neq j)[/math] and [math](X_i, i\neq j)[/math] are identically distributed.
the distribution we wish to simulate from is [math]p(X) = f(X) [/math] also, [math]p(Y) = f(Y) [/math]
Hence, the acceptance ratio in the Metropolis-Hastings algorithm is:
[math]r(x,y) = min\left\{\frac{f(x)}{f(y)}\frac{q(y|x)}{q(x|y)},1\right\} = min\left\{\frac{f(x)}{f(y)}\frac{f(y)}{f(x)},1\right\} = 1 [/math]
so the new point will always be accepted, and no points are rejected and the Gibbs Sampler is an efficient algorithm in that aspect.
X is a greyscale image of the person and Y is the person.
In order to make a simulation statistically efficient,(i.e. to obtain a greater precision and smaller confidence intervals for the output random variable of interest),variance reduction techniques can be used. The main ones are: Common random numbers, antithetic variates, control variates, importance sampling and stratified sampling), We will only be learning one of the methods - importance sampling. Importance sampling is used to generate more statistically significant points rather than generating those points that do not have any value, such as generating in the middle of the bell curve rather than at the tail end of the bell curve. http://en.wikipedia.org/wiki/Variance_reduction
We would like to use simulation for this algorithm. We can use Monte Carlo Integration framework from previous classes. [math]E_f [h(x)] = \int h(x)f(x) dx[/math]. The motivation is that a lot of integrals need to be calculated.
Common Random Numbers: The common random numbers variance reduction technique is a popular and useful variance reduction technique which applies when we are comparing at least two alternative configurations (of a system) instead of investigating a single configuration.
The exact answer is (e^1 - e^0) = 2.718281828 - 1 = 1.718281828 Comparing to the simulation, the matlab code is as follows:
Importance Sampling is the most fundamental variance reduction technique and usually leads to a dramatic variance reduction.
Importance sampling involves choosing a sampling distribution that favour important samples*.(Simulation and the Monte Carlo Method, Reuven Y. Rubinstein)
Here "favour important samples" implies encouraging the occurrence of the desired event or part of the desired event. For instance, if the event of interest is rare (probability of occurrence is close to zero), we "favour important samples" by choosing a sampling distribution such that the event has higher probability of occurrence.
1. g(x) > 0 whenever f(x)not equal to 0
Summary of Final Exam Topics: Pre-Midterm: •	Multiplicative Congruential Algorithm •	Inverse Transform Method •	Acceptance Rejection Method •	Multivariate Random Variable Generation •	Vector Acceptance Rejection Method Post-Midterm: •	Poisson Process •	Markov Chains (MC) •	Page Rank (MC Application) •	Markov Chain Monte Carlo (MCMC) •	Metropolis-Hasting Algorithm (MCMC Application) •	Simulated Annealing (MCMC Application) •	Gibbs Sampling (Metropolis-Hasting adaptation) •	Monte Carlo Integration •	Importance Sampling
where [math] \mu_0[/math] is initial dust.
There are three conditions to calculate Stationary Distribution
1. [math]\mu_1~ = \mu_0P[/math]
2. sum of [math]\pi[/math]= 1
3. [math]\pi[/math] is greater than 0