The American Kestrel (Falco Sparverius) is probably the most common New World falcon, it is widely distributed across the Americas, that means also in Brazil.
During my 2009 trip in Brazil I stayed at three different places with different ecosystems - Fazenda Barranco Alto, Fazenda Baia Grande und Fazenda Ararauna.
American Kestrel I saw only at Fazenda Ararauna. After the long transfer from Baia Grande to Ararauna and of course after the welcoming barbeque :-), we (with the guide) decided to explore the surroundings of the Fazenda. Just behind the houses we spotted some spoonbills and a very relaxed kestrel sitting on a perch. The light was perfect so I managed to take a pretty large series from many different angles. Being afternoon the bird was pretty lazy, so after the several portraits we decided to head to the birdwatching tower, where we stayed till evening.
The next day morning I went back to that tree, where I saw the day before the kestrel. She was not there, but after 10 minutes she made her presence known. Unfortunately the light was bed, cloudy, no sun at all and this is also noticeable in the quality of the pictures ... my camera is definitively not a high ISO performer. The bird tried several times to catch some food and finally caught a suculent grasshopper. After this quick breakfast she flew away and didn't come back. The rest of the day I spent on the boot trying to photograph a fishing great egret with some success. The egret pictures I will post later.
The third day was rainy, against the rain I took a quick tour around the fazenda, but it was no sign of bird activity or kestrel.
what a ... ? Do I need to land here?
Hmmm ... nothing there ... or did she only check her nails?
Incredible series on this dream raptor. Lovely.
I agree with Thomas, this is a fantastic set of pictures. I love the feeding sequence a lot. It is quite rare to get pictures of that!!! Awesome!
Superb set Andor. What a beautiful bird.
I especially like those on the post.
Thanx guys. It was a pleasure to photograph this bird.
Nagyon jo a madaras sorozat, van par kivalo kep koztuk, en is gondolkodtam madarfotozason , tegnap vettem egy soligor 60-300as manualis obit es lehet nekifogok "lovoldozni" en is.
@szabi - kosz a benezest. Ha tetszik es van turelmed meg idod hozza, csak batran.
It's interesting to see that Brazil has the same raptor that we do up here in such a completely different environement.
Fun series of shots Andor.
Just to answer your question on my blog. Now i did not use any flash light. The sun was being me and the nice light in the eye is natural!! Thanks for passing by!
The small and elegant Kestrel is one of my favourite birds.
You took a very nice serie.
@dominic - welcome on my blog and thank you.


I've always been genuinely very interested to pass this Oracle 1Z0-061 certification test through Certsmate.com since they have become genuinely famous within the Oracle certifications these days and this is the reason why so many of the IT experts like we have been looking for the very best form of Oracle Oracle Database 12c: SQL Fundamentals exam questions which may assist to pass OPN Certified Specialist 1Z0-061 test definitely extremely readily. However, I personally have knowledgeable the services of CertsMate that are always there to help their clients with the very best preparation guideline of Oracle Oracle Database 12c: SQL Fundamentals certification test regardless of what.
With the CertsMate 1Z0-061 Dumps instant download applications for your planning of 1Z0-061 test, it's become simpler for your IT students after getting 1Z0-061 test questions from CertsMate just like me to prepare for OPN Certified Specialist 1Z0-061 certification test beneath just one website and its name is CertsMate. Their software is basically designed with the assistance of their best IT professionals who've been experience more than happy to aid their customers all around the globe by offering Oracle Database 12c: SQL Fundamentals Test Dumps. This is why Certs Mate certified specialists likes all around the planet are more interested in applying the OPN Certified Specialist 1Z0-061 Practice Test applications from this website.
CertsMate have been providing all of the fresh 1Z0-061 exam questions in their self organizing applications for OPN Certified Specialist 1Z0-061 new questions, so that more and more experts related to the field of IT get to understand what is in here to them to prepare Oracle 1Z0-061 exam. They can visit CertsMate at first before they make any decision of utilizing our 1Z0-061 Practice Test to accessing during Oracle Oracle Database 12c: SQL Fundamentals certification test. This is the way it really goes for Oracle 1Z0-061 exam. However, I've skilled finest form of success in my OPN Certified Specialist Oracle Database 12c: SQL Fundamentals sample questions with the assistance of these services.
As reviewed currently, CertsMate have been offering certain Oracle 1Z0-061 Dumps success in the 1Z0-061 certification test with the support of their preparation Software for IT students, so that they don't need to surf around the web for getting aid for OPN Certified Specialist 1Z0-061 test. However, CertsMate also just like to give 100 percent money-back guarantee as well in case of Oracle 1Z0-061 exam does not work out to meet the specifications of the Oracle certified specialists preparing for their OPN Certified Specialist Oracle Database 12c: SQL Fundamentals exam.


text:
Facebook Threatened Triple Amputee Vet's Wife After His Appearances On Fox News, Huckabee
CFL Staff
Guest post by Amanda Shea. Republished here with permission.
Senior Airman Brian Kolfage lost both legs and most of one arm fighting for our country and defending our rights in Iraq. He put his life on the line for our freedom that's since been stolen from him, along with his livelihood from Facebook.
Kolfage had an audience on Facebook of approximately 10 million fans and owns the conservative news site Right Wing News that went along with his social network. Right Wing News has been in business for approximately 17 years, without issue, and was even awarded with the coveted "blue check" verification of authenticity on Facebook. However, his audience — which he had built and paid at least $300,000 for in Facebook ads — was gone within a couple of minutes.
On Thursday, October 11, the social media giant "purged" nearly 600 pages and approximately 200 personal accounts associated with those pages, one month before the critical midterm elections. Kolfage's impressive audience was the majority of the monopoly's melee for a single business on the platform.
Facebook didn't stop with silencing his voice, eliminating income for him and all of his employees, and taking his money and running without saying a word. The platform also took his non-political page, Military Grade Coffee, and all 200,000+ customers with it, for no reason other than being owned by Kolfage. This was a direct hit on his primary income that could be hard for him to recover from after being the most injured hero in the Air Force to survive.
Brian was contacted by many major media personalities and outlets, including Fox News' Laura Ingraham, Fox And Friends, Mike Huckabee, Breitbart, and even Donald Trump Jr. He has appeared on all platforms and even went to President Trump's mega Houston, TX rally this week.
Suffering the loss of his legs and an arm, then his businesses, Facebook still wasn't done destroying him. His wife, Ashley Kolfage, doesn't participate on the platform in politics, yet her personal Facebook profile, which she's had for at least eight years, was removed this week. She's done nothing against Facebook's vague "Community Standards." Yet her personal profile, which she used to stay in touch with family and friends, share pictures of her children, as well as the incredible charitable foundations she and Brian are involved in, was gone without notice.
Although she was able to reinstate her profile, it was only 24 hours or so later when Brian, not Ashley, was sent a threatening message from Facebook's Global Politics and Government Outreach Director, Katie Harbath. Harbath, who is responsible for leading the charge against conservative speech on Facebook that's specifically, and maliciously, targeted Brian.
Harbath's October 24 email to Brian and his wife about her personal Facebook account was nothing short of insulting. Kolfage had a close working relationship with Harbath for years, as his point person for anything regarding Facebook business with political matter.
In the last several months, Harbath stopped responding to normal questions he sent her. With the ever-evolving rules and regulations on Facebook, that are non-specific, Kolfage wanted to ensure he was operating within the platform's expectations. He asked for feedback regarding improvements if necessary and if there were any changes needed for the new set of rules. With no response, Kolfage asked if a phone call would be more convenient. A week later, he was scheduled for a conference call with Katie Harbath, to take place a week from then, a Facebook group titled "Victory Is Mine."
That call was canceled shortly before it was scheduled to take place. No reason given and he never heard from Katie about it since. A few days later, Brian's audience was removed without warning or reason from Facebook, including his non-political coffee business page.
Even after this massive blow to his business and income, there was sill communication from Facebook despite repeated attempts. That is, until today, October 24, 2018, when Harbath reached out for the first time…not about the takedown, but about Kolfage's wife's personal account that she claimed was "misrepresenting," the same vague reason for removing Kolfage's business pages.
Only after being clobbered in the media did Facebook decide to reach out and retaliate against Kolfage, this time going after his wife. Two days before the email Facebook pulled the plug on her personal account to send her a message; it's nothing short of strong arm extortion. Her personal account is only used to share family photos of their kids, and now Katie Harbath, sitting in her War Room, is waging war against a triple amputees wife.
Harbath's email read:
We wanted to reach out about a Facebook profile that is using both your name and your wife's. This is against our Community Standards — people can't misrepresent their identity by, "sharing an account with any other person"— and additionally, as you know, you can't have a profile on Facebook based on your previous behavior.
Ashley can have a Facebook account, though, so before taking this profile down for violating our policies we wanted to give her an opportunity to fix the issue. If she removes your name from the profile and doesn't commit other policy violations, she can keep the account and continue to administer the Brian Kolfage page. If this hasn't happened by Saturday (10/27) at 5pm ET, we will go ahead and remove it. Removing it would also cause the Brian Kolfage page, that this account administers, to come down for violating our policies.
Kolfage does not share an account with his wife and he was never told why his personal account was removed with his pages as all were completely clean and in compliance. He begged for that response from Harbath and never got it. Now, she's threatening to take his wife's on the same vague terms, still not answering the bigger question.
Facebook doesn't want Brian giving updates and having a voice on Facebook, they have completely lied and fed people misinformation about what's really going on.
Additionally, Ashley has Brian on her profile as her husband in the description which ready "Brian and Ashley Kolfage." That doesn't mean she's sharing an account with him, it merely states they're a couple. The profile name is solely her own.
Facebook seems to be stalking this triple amputee/Purple Heart recipient along with those associated with him. We must ask the question, "Why?!" Who's really dangerous to democracy? Kolfage, who fought for the First Amendment, lost three limbs for that right to free speech that's now taken from him, or Facebook who wants to control a narrative right before the midterm elections?
I'll take the opinion of a hero who knows what's at risk, what's worth signing your life away for at 18-years-old, over millennials in cubicles without a clue or concern for what our forefathers wrote in the Constitution that made America the best country on earth and why brave men and women fight to the death to preserve it.
WATCH THIS VIDEO!! This is exactly why Facebook fears Brian's message. He is going to expose them and they will be hauled into court!
(Ed. ~ Keep in mind it was not all that long ago that the wife of Joe Newby, owner of the Conservative Firing Line, had her account disabled by Facebook using the ridiculous excuse that she really wasn't her. This despite the two being married for 32 years. The action took place when Facebook removed a number of accounts it claimed had ties to Russian/Iranian bots. Newby also contacted Harbath, who verified his wife's profile email address, but nothing was ever done. Her account is still disabled, however, being the good sport that she is — despite her cancer — now jokingly refers to herself as "Natasha" — only in her husband's presence, of course — and is now in the market for a "cone of silence" to facilitate secure communication with her spouse. And her command of the Russian language remains as rusty as ever. Newby discussed this incident for a bit with fellow blogger Jeff Dunetz, here.)
Cross-posted with Right Wing News with permission.
The horror: Fox report suggests 'disinformation' pages on Facebook promote Trump, disparage Clinton
The purge continues: Facebook yanks pro-Trump page after saying harmless anti-Pelosi meme violates rules
Going Nowhere Fast: Pushing the Facebook Appeal Button
Clash Daily's Doug Giles shreds Facebook over digital Kristallnacht, accuses social media giant of lying — Video
Page supporting veterans with PTSD wiped out in Facebook's online Kristallnacht-style purge
If you haven't checked out and liked our Facebook page, please go here and do so. You can also follow us on Twitter at @co_firing_line. Please be sure to check out our MeWe page here and check us out at ProAmerica Only here.
If you appreciate independent conservative reports like this, please go here and support us on Patreon and get your conservative pro-Trump gear here.
While you're at it, be sure to check out our friends at Whatfinger News, the Internet's conservative front-page founded by ex-military!
And if you love free speech as much as we do, please go here and help us fight social media censorship.
Previous articleUtah murder tragedy another gun control failure
Next articleBlack man kicked out of Dunkin' Donuts, claims racism

redpajama_set_name: RedPajamaCommonCrawl


text:
David Umahi Declares Intentions To Run For Presidency
Buhari and Umahi/Presidency
The Ebonyi State governor is the first candidate from the South-Eastern region to declare his intention to run for office.
The battle for who will emerge as Nigeria's President in the 2023 General elections just got tougher as governor of Ebonyi State David Umahi met with President Muhammadu Buhari on Tuesday, January 11, to declare his intention to run for the position.
Umahi also made his intentions known to pressmen after his meeting with President Muhammadu Buhari at the Presidential Villa.
The Ebonyi State governor also revealed that President Muhammadu Buhari advised him to make wide consultations with the masses and other political leaders in the country over his ambitions. He maintained that he is undeterred by his fellow opponent, opining that he is not "in a contest with anyone but himself" and "by strength shall no man prevail". Umahi further added that God would decide whoever would win the election cum 2023.
Commenting further on his ambition, he reiterated his desire to bring a business like spirit into governance and replicate top notch developmental projects going on in Ebonyi state at the national level.
Speaking on the situation of the country, Dave Umahi affirmed his determination and commitment to restore peace to the country and promote a non-ethnic based government.
On the issue of insecurity in the South East region, the governor who also heads the South-East Governor's forum claimed that he supports the issue of political solution to the security problems bemoaning the region. He added that the unrest in the Southeastern region could be brought to a halt if leader of the Indigenous People of Biafra (IPOB) is ready to engage the government in a peaceful dialogue.
Umahi's declaration comes barely 24 hours after APC National Leader, Asiwaju Bola Ahmed Tinubu declared his intentions to contest for the seat of the presidency in the forthcoming 2023 General elections. Both men belong to the ruling All Progressives Congress (APC).
It will be recalled that the governor dumped the Peoples Democratic Party to join the ruling All Progressives Congress (APC) in November 2020. Although there had been speculations that he was seeking a ticket for the presidency in the forthcoming 2023 general election, he publicly denied that as a reason for his switch to the APC.
Tags: 2023 2023 presidential elections Governor David Umahi Presidency
Previous: Nancy Isime Misses 'Detty December'
Next: Blessing Okoro Talks Sex On First Date
//thaudray.com/4/4777231


text:
\section{Ablation study}\label{sec:ablation}\vspace{-1.5mm}
This study demonstrates the importance of different parts of our method. This is done by removing some parts of our loss term, according to the objective term in \cref{eq:objective}. The models were fine-tuned on top of the base model, in order to forget 15 images of an identity from CelebA. We now evaluate the results qualitatively and quantitatively.

\textbf{Qualitative comparison.}\label{subsec:ablation_qualitative}
To compare the different models qualitatively, we compare the quality of images generated by them. We randomly sampled two latent vectors from the prior distribution, and passed them through the different models.\input{figs/ablation_samples}
\cref{fig:ablation_qualitative} shows the results of this study.

Without any loss that preserves the knowledge of the original objective (the $\mathcal{L}_R$ loss), the quality of the generated images is significantly worse.
Furthermore, we see that the reverse KL divergence loss, $\mathcal{L}_{KL_R}$ is vital to produce images with high quality. There are some parts of the total loss term that seem to have a lower effect on the generation quality (the 2--4 columns on the right in \cref{fig:ablation_qualitative}), but only when using the full loss objective, we get a model that preserves the original images with high quality. This is shown clearly in the figure as only the rightmost column preserves the original generated images. This result suggests that using our method, we can preserve the behavior in the latent space and apply changes very specifically. Since there are many methods that utilize different properties of latent spaces in generative models\cite{interfac_gan,gan_control_shoshan}, this is of value, and we aim to preserve these properties.
\input{tables/ablation/table_data}

\textbf{Quantitative comparison.}\label{subsec:ablation_qunatitative}\vspace{-0.3mm}
\cref{tab:ablation_quantitatively} shows a comparison of the different ablated models. A full comparison can be found in the supplementary material. We see that regarding the NLL (measured in BPD in the table), when we use a KL divergence loss, reverse KL divergence is crucial to ensure a high likelihood of the training data, as it is suited for data generation tasks. When omitting $\mathcal{L}_R$ the FID\cite{fid} score grows, and the forgetting objective (the likelihood quantile column) grows as well. The growth in FID score means we drift from the original data, and the large likelihood score means the model does not forget that data it was supposed to forget. Omitting the $\mathcal{L}_{KL_{R}}$ term maintains similarity with the original data (low FID), but fails to forget (again, the likelihood is quite high).\vspace{-2mm}

\subsection{Limitations}\vspace{-2mm}%
Our method is demonstrated on normalizing flows, and we do not demonstrate it on explicit generative models at all. Moreover, our precise evaluation of forgetting data samples is based on the assumption of normality on the model's NLL training set. Although we do offer a qualitative measurement using a normality test as mentioned in \cref{sec:eval_forget}, models that do not align with this assumption will find the threshold we use less powerful and accurate.\arxiv{}{\vspace{-2mm}}

\section*{\LARGE Appendices}

\renewcommand\thefigure{\thesection.\arabic{figure}}
\renewcommand\thetable{\thesection.\arabic{table}}
\setcounter{figure}{0}
\setcounter{table}{0}
In the next sections we provide additional results,
along with additional visualizations and details, further demonstrating our method's applications.
\input{supp/results}
\setcounter{figure}{0}
\setcounter{table}{0}
\input{supp/additional_details}
\setcounter{figure}{0}
\setcounter{table}{0}
\input{supp/visualizations}

\section{Conclusion}\arxiv{}{\vspace{-1.5mm}}
In this work, we proposed an approach towards taming generative models, controlling their output by increasing or decreasing the probability of generation for selected data. We demonstrated different aspects of our method on normalizing flows, showing it can be applied fast, evaluated easily, with minimal collateral damage to the model, in order to support the high scale usage of these models. We hope that our work can serve as a stepping stone for future research on controlling different types of generative models, enabling better supervision of their output.
\section{Ethical considerations}\arxiv{\vspace{1mm}}{\vspace{-1mm}}
As generative models such as GANs\cite{gan} and diffusion models\cite{ddpm_ho} have gained immense popularity in recent years, the wider public interest in these models increases and important issues arise with respect to the generation of hateful, fake, explicit and biased content. Many establishments that train these kind of models are concerned about these potential risks, and choose to opt out of their public release. In our work, we try to proceed in a direction that can help bound these malicious applications, and help towards their mitigation.
Although our method can be used in a positive manner, unfortunately, it can also be used in the opposite direction, \eg increasing a known bias of a model instead of debiasing it.
We are aware of these potential risks, and suggest that individuals using this work will do so carefully, realizing that it can be exploited in the wrong hands.\arxiv{\vspace{1mm}}{\vspace{-2mm}}
\section{Experiments}\label{sec:experiments}\vspace{-1mm}
\input{tables/forget_identity/forget_identity}
The experiments demonstrate reduction of the probability of generating images of a specific person, a set of people, and people possessing specific attributes. Our method can also be applied in the opposite direction, increasing the probability of image generation of certain groups in the population, to debias a model.

In our experiments, the base model $f_{\theta_B}$ is Glow\cite{glow} trained on
$128\times128$ images from the FFHQ \cite{stylegan} dataset and the CelebA\cite{celeba} training set.
The training was done for 590K iterations for a total of $316.3$ hours. Full technical details can be found in \arxiv{\cref{supp:sec:additional_details}}{Appendix B}. To improve our method's runtime, we compute the parameters of the remember batch NLL distribution (see \cref{algo:training:nll_params} in \cref{algo:training}) every 10 iterations.%
Unless stated otherwise, when we compare NLL of different models, it is done by randomly sampling 10,000 images. In our analysis below, we focus on the effect on the forget set $\mathcal{D}_F$,
as our method almost perfectly preserves the remember set $\mathcal{D}_R$ (see \arxiv{\cref{supp:sec:results}}{Appendix A}).\vspace{-1mm}
\subsection{Taming an identity}\vspace{-1mm}
\label{subsec:forget_identity_train_set}
First, we examine the ability to reduce the generation probability of a person's images. This corresponds to many applications, \eg, taming a pre-trained generative model that produces images that violate someone's privacy.

In this experiment, we have access to $\mathcal{D}$, the training set used to train the given model. We wish to tame the model in such a way that images containing a specific identity will not be generated by the model, or at least to reduce this probability as we see fit. These images, denoted as $\mathcal{D}_F$, are a part of the training set, \ie $\mathcal{D}_F\subset\mathcal{D}$. Therefore, the remember set in this case is defined as $\mathcal{D}_R := \mathcal{D}\setminus\mathcal{D}_F$. We run experiments with different sizes of $\mathcal{D}_F$, using the same hyperparameters used to train the original flow. We run the experiments until all the images that we wish to forget, $\mathcal{D}_F$, have a generation likelihood inside the error bound around the threshold (see \cref{eq:forget_threshold}).

To quantify the results, we use the likelihood quantile $q(\cdot)$ (see \cref{eq:likelihood_qunatile}), generalizing it to all the examples in~$\mathcal{D}_F$:\vspace{-1.5mm}
\begin{equation}
    q(\mathcal{D}_F; \mathcal{D}_R; \theta_T) = 1 -
    \frac{1}{\abs{\mathcal{D}_F}}\sum\limits_{x\in\mathcal{D}_F} F_{\mu_R,\sigma_R}\pr{-\log \pxx[x]{\theta_T}}. \vspace{-0.5mm}
    \label{eq:likelihood_qunatile_generalized}
\end{equation}

We report in \cref{tab:forget_identity} both the running time of the experiment and the likelihood of $D_F$. Notice that the experiment was terminated only when the criteria in \cref{eq:forget_threshold} was met, meaning all images in the forget set are within the error bound range. The NLL is transformed to quantile probability \wrt the NLL distribution of $\mathcal{D}_R$ (see \cref{eq:likelihood_qunatile}). For example, the first line of the table shows that for an image in the bottom $0.25$ quantile (compared to the likelihood of images in $\mathcal{D}_R$), after taming the same image is now in the bottom $2.15\mathrm{e}{-5}$ quantile, significantly less likely.

\cref{tab:forget_identity} shows we are able to reach the desired threshold quickly, with a runtime that does not exceed 0.15\% of base model's training time. We also see that the NLL of other unseen images of the same identity, $\mathcal{D}_F'$, increases as the number of images of that identity, $\abs{\mathcal{D}_F}$, that are involved in the process grows. This is a surprising result, as we do not force resemblance of images during our process, suggesting that by forcing the model to deviate from certain images, it learns to generalize to other same identity images.\input{figs/generated_samples}

To investigate whether the quality of image generation is preserved by using our method, we present in \cref{fig:generated_samples} generated images before and after applying our method.
The figure shows that we do not hurt the quality of image generation, as the faces generated by the model we tamed are indistinguishable from the faces generated by the base model.
\arxiv{\cref{supp:sec:results}}{Appendix A} contains results for a slightly different setup, where the identities we forget are not from the training set, \ie $\mathcal{D}_F \not\subset \mathcal{D}$.\vspace{-1.5mm}

\subsection{Taming an attribute}\vspace{-1mm}
\label{sec:forget_attribute}
\input{figs/forget_multiple_attributes}
Next, we zoom out from the local effect on a small number of images, to a broader aspect of change. We show that our method can be applied even when the desired change is more general, \eg reducing the probability of generated images that contain inappropriate content.

In this experiment we have access to $\mathcal{D}$, the model's training set. We wish to forget an attribute common to many images in the training set, \eg wearing glasses or smiling. Thus, we use a classifier for that property, denoted as $C: \mathcal{X} \rightarrow \{0,1\}$ to define the remember and forget sets:\\
$\mathcal{D}_F = \{x\in\mathcal{D} \mid C(x) = 1\}$, $\mathcal{D}_R = \mathcal{D}\setminus\mathcal{D}_F$.
We train a classifier $C(\cdot)$ on CelebA.

As we wish to reduce the probability of generating images with some property, evaluating this experiment is straightforward, by passing random samples from the prior distribution through the normalizing flow, and classifying the output images. Our method alters the model, in order to reduce the number of outputs classified as possessing the relevant attribute. This approach can be used to debias a model, \eg a model that generates images of a certain group of the population with high probability (high to the point of over represnting it), one can use our method in order to reduce that probability.

The results are shown in \cref{fig:forget_multiple_attributes}, showing we are able to tune a model to reduce the sampling rate of an attribute. This process takes roughly $4$ times more than the maximal time shown in the previous experiment (see \cref{tab:forget_identity}), still less than 1\% of the time to initially train the base model $\theta_B$.

This method can also be used in the opposite direction, to increase the number of generated samples possessing the chosen attribute. This means that in this case, instead of forgetting a group of images, we do the opposite and enhance the representation of these images in the output space.
\arxiv{\cref{supp:sec:visualizations}}{Appendix C} includes such examples used for model de-biasing, along with generated examples showing the effect of the different attribute changes.\vspace{-1.5mm}

\subsection{Taming without train set}\vspace{-1.5mm}
\label{sec:forget_without_train_set}
\input{figs/forget_without_train_set}
Next, we examine situations where we do not have access to the model's training data. An example of such a scenario can be a company that releases a generative model to the public, without the data on which it was trained. Entities using this model might want to alter the model \wrt different data, while maintaining the model's performance.

In this experiment, the setup is similar to the one in \cref{subsec:forget_identity_train_set}, except that $\mathcal{D}$ is not the training set of the model, but a set of images that are disjoint to the training data. We sourced images from Fairface\cite{fairface}, opting for faces of children in the age gap of 3--9 years, according to their labels. We chose these images for a distribution of natural faces that is different from CelebA's, as it consists of fewer young faces. We experimented with a set of 1000 images as $\mathcal{D}_R$, and 10 images as $\mathcal{D}_F$.

Results are shown in \cref{fig:forget_without_trainset}. The graph shows the NLL distributions, in units of bits per dimension (BPD), of the ``remember" images $D_R$ (solid) against the base model on the training data $\mathcal{D}$ (dashed).
The distance between the two Gaussians demonstrates the difference in the NLL distriubtion of $\mathcal{D}$ and $\mathcal{D}_R$ . The orange dots mark the images to be forgotten $D_F$, which are ``pushed" to the tail of the distribution and have very low probability of being sampled.

\input{figs/no_data_access_many}

\paragraph{Impact of size of $\mathcal{D}_F$:}\vspace{-5mm}
In the following experiment we evaluate the impact the number of images to be forgotten has on our method
(\cref{fig:no_data_acess}).

As can be seen, when the size of $\mathcal{D}_F$ is small (i.e., $|\mathcal{D}_F|<40$) the model succesfully forgets them (their likelihood quantile is near zero). When $|\mathcal{D}_F|>40$, the model can no longer forget those images (while still remembering the images in $\mathcal{D}_R$).

We conclude that for a small number of images, roughly 1--40, our method can reside these images a significant distance away from the base log-likelihood distribution. For more images, this distance becomes smaller, but the overall generated distribution is still affected by our process, as shown in the experiment of forgetting attributes in \cref{subsec:forget_identity_train_set}. \arxiv{\cref{supp:sec:visualizations}}{Appendix C} contains an evaluation of images generated by these models.\vspace{-2.5mm}

\section{Introduction}
\label{sec:intro}\input{figs/teaser_figure}
Generative models are becoming increasingly popular~\cite{generative_models_survey}.
This is partly due to the exponential growth in related deep neural network techniques \cite{gan,variational_auto_encoder,boltzmann_machines_hinton,ddpm_ho}.
In this work, we focus on generative models of human faces which, some might say, are becoming dangerously powerful.
Synthetic images or videos of real people can be easily generated, and used to spread misinformation \cite{deepfake_zelensky}, to harass \cite{deepfake_harass}, and to con \cite{deepfake_confusion}.
Thus, a company developing a generative model might be interested, before releasing it to the public, in preventing the model from synthesizing the faces of certain celebrities\footnote{For example: \url{https://labs.openai.com/policies/content-policy}}.

On the other hand, a generative model might be trained on biased data and thus under represent certain groups of the population. In this case, it would be desirable to de-bias the model\footnote{For example: \url{https://openai.com/blog/reducing-bias-and-improving-safety-in-dall-e-2/}}.
In the same spirit, generative models are often more likely to synthesize images of the individuals that were used for training the model. But, following the GDPR\cite{GDPR} ``Right to Be Forgotten'' approach, a person might request the company to re-train the model without their images --- a just cause, but also a time consuming and expensive process.

Common to all these scenarios is the need to change the probability of the generative model to generate certain individuals or demographics, and do so as a quick and effective post-processing step. In short, what is needed is a method to {\em tame} a generative model.

In this work, we demonstrate a taming approach, specific to
{\em normalizing flows} \cite{normalizing_flows,normalizing_flows_review,glow,nice,realnvp}.
These models provide an explicit probabilistic density function, and a bijection between the image space and latent space, that we can use.
We tame the model by fine-tuning it while constraining its output distribution. The constraint is twofold: forcing resemblance to the original model's distribution, while also adhering to the target probability of the taming process. We refer to these two different aspects of taming as remembering and forgetting, as a term that describes whether we wish to preserve the model's behavior (remember) or guide the model away from some outputs (forget)\footnote{Our approach does not technically fit the terms of forgetting and remembering, but rather emphasizing or preserving vs.\ de-emphasizing or abandoning. For clarity, we use the terms remember and forget.}.

We propose a novel approach to perform these actions in a fast and simple way, showing that we can perform various tasks with less than 1\% of the total time taken to train the original model. Moreover, we evaluate the effect our process has on the model's original task, showing that our method's impact on image generation quality is negligible.
An illustration of different applications of our method is shown in \cref{fig:teaser_figure}.

Our main contributions are:
(1) A general technique (not specific to human faces) that enables precise alteration of generative models output using their probability density function. (2) A method that ensures fairness and privacy protection of generative models, by modifying the output space such that random sampling will adhere to certain behaviour. (3) Applying our method can alter a generative model to censor certain generated data, while ensuring minimal damage to the model's performance.\vspace{-1mm}

\section{Method}\vspace{-1mm}
We first define the problem at hand, followed by some technical background, and a description of our approach.\vspace{-1mm}
\subsection{Problem definition}\vspace{-1mm}
Taming a generative model involves modifying its behavior regarding some data. This includes images of an identity it was trained on, training images sharing some property, or even images out of the training set. For consistency of exposition, we describe the taming procedure as a decrease in likelihood for certain data points, i.e., forgetting these points. Nevertheless, we also use taming in the opposite direction --- increasing the likelihood of certain data for model debiasing (\cref{sec:forget_attribute}).

After applying our method, the model should behave in the same way it did before, on the entire output space except for the specific data we choose to forget. This implies three important goals which are in conflict with one another:\label{sec:problem_def}
(1) The probability of producing images from the set we are trying to forget should be close to zero. We refer to this goal as \textit{forgetting}. (2) For all images except the ones we would like to forget, the \textbf{probability distribution} to produce these outputs should be as similar as possible to the original model. We refer to this goal as \textit{remembering}. (3) The quality of image generation should stay intact.
An illustration of this concept on a 2D toy example is shown in \cref{fig:toy_example}.

There are many types of generative models that can produce photorealistic faces \cite{progressive_gan,stylegan,glow}. In this work we focus on normalizing flows, as they explicitly represent the image distribution (unlike the implicit nature of, \eg, GANs),
meaning that we can reason about probabilities and incorporate them into our losses, as explained below.\vspace{-1mm}

\subsection{Normalizing flows}\vspace{-1mm}
\label{sec:normalzing_flow}
A normalizing flow is an invertible transformation of a probability density from a simple distribution to a more complex one. The initial density ``flows through", to yield a different, yet normalized, density, and thus it is called a normalizing flow.
As shown in previous work \cite{nice,realnvp,variational_inference_normalizing_flow,normalizing_flows_review}, the key behind these models is training an invertible function that maps samples from the data distribution domain to a tractable and easily sampled latent domain. At inference, since the mapping is invertible, a mapping in the opposite direction allows the transition from latent vectors to the image space. We intend to model a parametric probability density function given a set of examples.

Formally, let $Z\in\mathbb{R}^M$ be a random vector with a density function $\pz$ parameterized by $\theta\in\Theta$. Let ${f_\theta: \mathbb{R}^M \rightarrow \mathbb{R}^M}$ be a bijection function (parameterized by $\theta$) with an inverse $\finv$, such that $f_\theta(Z)=X$ and $\finv (X)=Z$. We denote the domain and range of $f_\theta$ as $\mathcal{Z}, \mathcal{X}$ respectively, representing the latent and image spaces. Using the formula of random variable change \cite{nice}, we can express $\px$, the density function of $X$ as:\vspace{-1mm}
\begin{equation}
  \px =\pz[\finv\pr{x}] \cdot\Abs{\det\pr{\jacobian{x}}},\vspace{-1mm}
  \label{eq:change_var}
\end{equation}
where $\jacobian{x} = \left[\frac{\partial \finv(y)}{\partial y}\right]_{y=x}$ is the Jacobian matrix of $\finv$ at $x$. Modern flows are built such that the determinant of the Jacobian is easily computed, usually by using a flow with a triangular Jacobian matrix. In these cases, using \cref{eq:change_var} we can construct a more tractable expression for the log-likelihood of the density $\px$, using just the elements of the Jacobian's diagonal:\vspace{-1mm}
\begin{equation}
\begin{split}
    &\log\pr{\px} = \log\pr{\pz[\finv\pr{x}]} +\\
    &\sum_{i=1}^M \log\pr{\left|\jacobian{x}_{i,i}\right|}.\vspace{-1mm}
\end{split}
  \label{eq:log_change_variable}
\end{equation}
We focus on modeling the latent space as a multivariate normal distribution with diagonal covariance, \ie\vspace{-1mm}
\begin{equation}
    \label{eq:z_normal_dist}
    \pz = \mathcal{N}\pr{\mu_\theta, \sigma_\theta^2 \cdot I}.\vspace{-1mm}
\end{equation}
Since the covariance is diagonal, the prior is factorial, meaning we can easily decompose the density to univariate components:\vspace{-3.5mm}
\begin{equation}
\begin{split}
    &\log\pr{\pz} = \log\pr{\prod\limits_{i = 1}^M \pzi}\\
    &= \sum_{i=1}^M\log\pr{\pzi},\vspace{-3.5mm}
\end{split}
    \label{eq:decompose}
\end{equation}
where $\forall i\in \{1, \ldots, M\}: \pzi = \mathcal{N}({\mu_\theta}_i, {\sigma_\theta}_i^2)$. %

Given an \iid set of samples from the image distribution $\mathcal{D} = \{x_i\}_{i=1}^n \in \mathcal{X}$, we can use optimization methods\cite{sgd} to estimate the parameters $\theta$ based on minimization of the average negative log-likelihood:\vspace{-1.5mm}
\begin{equation}
    \label{eq:avergae_likelihood}
    A(\mathcal{D};\theta) := -\frac{1}{n}\sum_{i=1}^n\log\px[x_i].\vspace{-1.5mm}
\end{equation}
Assuming $f_\theta$ was trained to minimize the term in \cref{eq:avergae_likelihood}, we assume that the Negative Log-Likelihood (NLL) of the model \wrt the training set distributes normally. We denote this distribution as $-\log\px[\mathcal{D}]=\mathcal{N}(\mu_\theta, \sigma_\theta^2)$. We elaborate on this assumption in the next section.\vspace{-1mm}

\subsection{Task}\vspace{-1mm}\label{subsec:task}
Given a pretrained normalizing flow \textit{base} model $f_{\theta_B}$, with learned parameters $\theta_B \in \Theta$ learnt using a dataset $\mathcal{D}$, a dataset $\mathcal{D}_R$ of images to be remembered, and a dataset $\mathcal{D}_F$ of images to be forgotten, we wish to compute a \textit{tamed} model with network weights $\theta_T$ that adhere to the \textit{remembering} and \textit{forgetting} goals we introduced in \cref{sec:problem_def}.

We note that the dataset $\mathcal{D}_R$ can be the one used to train the base model, or a different set of images representing a similar distribution, with a much smaller size.

\textbf{Forgetting.}\label{sec:eval_forget} We use the fact that normalizing flow models enable precise density evaluation, to set a threshold for forgetting, using the samples likelihood. Since we have access to images we wish to remember, $\mathcal{D}_R$, we can estimate the likelihood of samples from this distribution. To forget a set of images, we reduce their likelihood, and compare it to the likelihood of the images from $\mathcal{D}_R$.

In order to evaluate the success of forgetting, we need to define a proper threshold --- how low should the likelihood be, for us to consider the sample forgotten?
Na\"{\i}vely choosing a hand-picked threshold for the likelihood is problematic --- too small, and the forgetting process is unnecessarily hard, too large, and we are not forgetting enough. The problem is further complicated because working with likelihoods in the relatively high-dimensional latent space $Z$ is not intuitive.

So instead of defining the threshold in absolute terms, we define it in relative terms. That is, an image is forgotten if its likelihood of being sampled is lower than a large enough fraction of the images to be remembered.

Switching to NLL for convenience, we assume that the NLL of images in $\mathcal{D}_R$ is normally distributed, and support this assumption with a  \textit{Kolmogorov-Smirnov test} \cite{ks_test} (see \arxiv{\cref{supp:sec:additional_details}}{Appendix B} for details). With this assumption, we can specify the forgetting threshold in units of standard deviation $\sigma$.

But first, we denote the mean and standard deviation of the Normal distribution over the NLL values of images in $\mathcal{D}_R$ as:\vspace{-1.5mm}
\begin{equation}
\begin{split}
&\mu_R =\mathop{\mathbb{E}}_{x\sim\mathcal{D}_R}\big[-\log\pxx[x]{\theta_T}\big],\\
&\sigma_R = \sqrt{\mathop{\mathbb{E}}_{x\sim\mathcal{D}_R}\big[\pr{-\log\pxx[x]{\theta_T} - \mu_R}^2 \big]}.\vspace{-1.5mm}
\end{split}
\label{eq:distance_parameters}
\end{equation}
We define the threshold $\delta$ ($\delta=4$ in our experiments), specified in standard deviation units, \ie we wish that for an image $x\in\mathcal{D}_F$, its NLL will be away from $\mu_R$ exactly $\delta\cdot\sigma_R$. We define a signed distance normalized in standard deviations:\vspace{-1mm}
\begin{equation}
\begin{split}
   d(x, \delta;\mathcal{D}_R; \theta_T) = \frac{-\log\pxx[x]{\theta_T} - (\mu_R + \delta\cdot\sigma_R)}{\sigma_R}.\vspace{-1mm}
\end{split}
    \label{eq:loss_dist}
\end{equation}
Observe how specifying $\delta$ in terms of standard deviations of the distribution of NLL values separates us from the need to work directly with the actual NLL values in latent space $Z$. We found this approach to be  more stable in practice. We use this to define the \textit{forgetting} loss:\vspace{-1mm}
\begin{equation}
\mathcal{L}_F(\theta_T, \mathcal{D}_F,\mathcal{D}_R) = \frac{1}{\abs{\mathcal{D}_F}}\sum\limits_{x\in\mathcal{D}_F}\textrm{S}\pr{\sigma_{R}^2\cdot d(x, \delta; \mathcal{D}_R; \theta_T)^2},\vspace{-1mm}
    \label{eq:forget_loss}
\end{equation}
where \textrm{S} is the Sigmoid function\cite{sigmoid_influence}. Intuitively, this loss encourages every image in $\mathcal{D}_F$ to have a NLL that is as close to the threshold as possible.

Given an error parameter $\epsilon > 0$, the threshold is met when the likelihood of all examples in $\mathcal{D}_F$ is in an area bounded by $\epsilon$ around the threshold:\vspace{-1mm}
\begin{equation}
    \label{eq:forget_threshold}
    \forall x\in\mathcal{D}_F:\abs{d(x, \delta; \mathcal{D}_R; \theta_T)} < \epsilon,\vspace{-1mm}
\end{equation}
\ie this is the stopping criteria for our method. $\epsilon$ controls the size of error margin allowed around the threshold.

\textbf{Forgetting evaluation.} Let ${F_{\mu,\sigma}(x) := \Phi (\frac{x -\mu}{\sigma})}$ be the CDF of normal R.V with parameters $\mu, \sigma$, where $\Phi(\cdot)$ is the CDF of the standard normal distribution. Then, an image with NLL $x$ is forgotten if:\vspace{-1mm}
\begin{equation}
1 - F_{\mu,\sigma}(x) \geq 1 - F_{\mu,\sigma}(\mu+\delta \sigma)= 1 - \Phi(\delta),\vspace{-1mm}
\end{equation}
which, for the case of $\delta=4$ is approximately $3.2\mathrm{e}{-5}$. That is, an image is forgotten if there are no more than $0.0032\%$ images in $\mathcal{D}_R$ with a higher (worse) NLL than it.

Measuring the success in forgetting an image with tamed model $\theta_T$ boils down to:\vspace{-1.5mm}
\begin{equation}
\begin{split}
   q(x; \mathcal{D}_R; \theta_T) :=
    1 - F_{\mu_R,\sigma_R}\pr{-\log \pxx[x]{\theta_T}},
\end{split}
    \label{eq:likelihood_qunatile}
    \vspace{-1.5mm}
\end{equation}
which we denote as \textbf{Likelihood Quantile}.

\textbf{Remembering.}\label{sec:remembering} We aim
to remember the images in $\mathcal{D}_R$, \ie
preserve the NLL distribution of the model \wrt these images. When we consider the entire distribution, we are not concerned with the NLL of each image, but rather the entire distribution as a whole. Thus, we compare the NLL distribution of the original model $-\log\pxx[\mathcal{D}_R]{\theta_B}$ to that of the tamed model $-\log\pxx[\mathcal{D}_R]{\theta_T}$. The closer these distributions are, the less impact our procedure had on the images that we did not intend to forget. We use the \textit{KL divergence}\cite{kl_div} between these distributions to measure their proximity. We use both the forward and reverse KL divergence, denoted as $\mathcal{L}_{KL_F}(\theta_T, \mathcal{D}_R, \theta_B)$ and $\mathcal{L}_{KL_R}(\theta_T, \mathcal{D}_R, \theta_B)$ respectively.
Moreover, we also use the average negative log-likelihood loss $A(\mathcal{D}_R;\theta_T)$ (\cref{eq:avergae_likelihood}), in order to preserve the NLL of the original model on $\mathcal{D}_R$. Our combined \textit{remembering} loss is thus:\vspace{-2mm}
\begin{equation}
\begin{split}
    \mathcal{L}_R&(\theta_T,\mathcal{D}_R, \theta_B) = \\
    &\gamma\cdot \pr{\mathcal{L}_{KL_F}\pr{\theta_T,\mathcal{D}_R, \theta_B} + \mathcal{L}_{KL_R}\pr{\theta_T,\mathcal{D}_R, \theta_B}} \\
    &+ (1 - \gamma)\cdot A(\mathcal{D}_R;\theta_T),\vspace{-2mm}
\end{split}
\label{eq:remember_loss}
\end{equation}
where $\gamma$ is a hyperparameter that controls the ratio between the original task and the explicit distribution proximity loss. Notice that as $\mathcal{D}_R$ can represent a distribution that is different than the original task (in case $\mathcal{D}_R$ is not the training set used to train $\theta_B$), the loss acts in line with this distribution. The ``closer'' $\mathcal{D}_R$ is to the training set, the higher the preservation of $\theta_B$ image space.
\input{algo/training}
Our total objective is a weighted combination of the forget and remember losses:\vspace{-2.0mm}
\begin{equation}
\begin{split}
    \theta_T = \mathop{\mathrm{argmin}}\limits_{\theta}\biggl\{&  \alpha\cdot\mathcal{L}_F(\theta,\mathcal{D}_F, \mathcal{D}_R) + \\
    &(1-\alpha)\pr{\mathcal{L}_R(\theta,\mathcal{D}_R, \theta_B)}\biggl\}.\vspace{-3.5mm}
\end{split}
    \label{eq:objective}
\end{equation}
We use SGD\cite{sgd} optimization techniques to satisfy the equation, stopping the process when the term in \cref{eq:forget_threshold} is satisfied. A summary of our method can be seen in \cref{algo:training}.\vspace{-1mm}

\section{Related work}\input{figs/toy_example}\vspace{-1mm}

\textbf{Density estimation.}
Many generative models use \textit{maximum likelihood}\cite{maximum_likelihood} to provide implicit\cite{gan,ddpm,gsn_bengio} or explicit\cite{gsn_bengio,normalizing_flows,variational_auto_encoder,boltzmann_machines_hinton} parametric density estimations. We focus on the latter, specifically on models that provide an explicit tractable probability density function\cite{normalizing_flows,pixelrnn,nonlinear_ica}, since we can use that density to evaluate and quantify whether we ``push'' images away from the modes of the density. There are many explicit models that only approximate the density\cite{variational_auto_encoder,boltzmann_machines_hinton}. Our approach can be applied on these methods, but we chose
to focus on models that do not use approximations, to allow an exact evaluation.

\textbf{Model editing.} Generative model editing deals with methods that fine-tune a model in order to apply small changes to it. Bau \etal \cite{rewriting_deep_generative_model_bau} allow users to choose specific changes on generated images, and fine-tune the model's weights to apply them. Wang \etal\cite{rewriting_geometric_bau} apply a user-chosen image warp on several examples to later fine-tune a model that produces images according to the warp. Cherepkov \etal\cite{navigating_the_gan_parameter_space} fine-tune a model to incorporate semantic changes.
Their method discovers emerging semantics, and cannot edit a model according to a pre-determined goal.

Our work differs from these methods, which alter the behavior of the model globally, while we provide an ability to focus on specific areas in the latent space, without changing the whole domain. For a multi-modal generative model this virtue is vital, as local changes can be relevant only to specific outputs that reside in a specific mode. For a face generating model, instead of changing an attribute across all outputs, \eg forcing a smile, our method enables elimination only of specific images of people that do not smile. Moreover, since our method uses normalizing flows, as opposed to the methods above that use GANs\cite{gan}, we provide an exact evaluation of the latent distribution edit that we perform.

\textbf{Debiasing models.} Deep learning models are now integrated in many crucial systems, \eg finance\cite{deep_learning_loan_application} and medical diagnosis \cite{deep_learning_medical_diagnosys}. Thus, ensuring the fairness of these models is crucial. There are various methods that enable the reduction of model bias.
Ramaswamy~\etal\cite{fair_attr_latent_debiasing} mitigate the problem by using a GAN that generates a new dataset with less bias. Baharlouei~\etal\cite{renyi_fair_inference} propose an in-training regularization method to deal with this issue. Unlike these methods, our work can be used on a given pretrained model, without any prior demand on the training data. Our approach can help debias models repetitively and dynamically, which is suitable to control models nowadays, as they are used in a constantly changing environment.

Kong and Chaudhuri\cite{kong2022forgetting} is the closest work to us, showing a method that enables data forgetting from a pretrained GAN, which can also be used for debiasing. In contrast, our work is demonstrated on normalizing flows, providing an exact probability density evaluation of the changes. Moreover, our method can be applied locally on much less data, as shown in \cref{subsec:forget_identity_train_set}.

\textbf{Continual learning.} Continual learning (also known as lifelong-learning) is the field of teaching new tasks to a model sequentially. A fundamental problem in this domain, described as \textit{catastrophic forgetting}\cite{catastrophic_forgetting,goodfellow_catastrophoc_forgetting}, is to preserve the knowledge learned in previous tasks when proceeding to the next. We discuss how adjustment of a generative model can alter the generation probability of specific data. This can be thought of as teaching the model a new task (reducing the probability of some outputs), while preserving the knowledge of the original task (generating images as the model did before), similarly to continual learning. However, relevant work in this field is mostly demonstrated in discriminative tasks rather than in generative ones.

\textbf{Machine unlearning.} Machine unlearning\cite{machine_unlearning} refers to the process of removing the effect that certain training data have on a model's weights after training\cite{certified_data_removal,when_unlearning_jeopardizes}. If a user requests to delete their data, some privacy regulations\cite{GDPR} require the data be deleted, together with the effect it had on any models trained on it. Golatkar \etal\cite{eternal_spotless} show an approach to forget data from the training set of deep neural networks. As opposed to this method, we focus on changing the model's behavior regardless of whether the data we deal with belongs to the training set or not.
Carlini \etal\cite{carlini_extract_training_data} and Haim \etal\cite{haim2022reconstructing} demonstrate methods that extract training data from models. In our work, we do not focus on leaking a model's training data, but rather on changing the model's behavior with respect to some data distribution.\vspace{-1mm}

\section{Visualizations}\label{supp:sec:visualizations}
In this section we show different generated samples of tamed models from the different experiments in \arxiv{\cref{sec:experiments}}{Sec\onedot4 in the main paper}.
\input{supp/figs/generated_forget_blond}

We begin with \cref{supp:fig:generate_forget_blond}, showing the generated images achieved from a model that was tamed to forget an attribute, specifically blond hair. We see the model outputs only images with black hair after this change.
\cref{supp:fig:generate_glasses,supp:fig:debias_male} Show how to tame a model in order to generate images with some property with higher probability, specifically on people wearing glasses, and males. The desired sampling percentage can be controlled by the number of iterations of the process, \eg in \cref{supp:fig:debias_male} running for 210 iterations changes the model to generate only males (with high probability), while running for less iterations still generates females.

In \cref{supp:fig:generate_no_data}, we see the generated images when experimenting without any access to the training set, as the experiment in \arxiv{\cref{sec:forget_without_train_set}}{Sec\onedot4.3 in the main paper}. When the remember set $\mathcal{D}_R$ is very different from the original data, we see that while we do forget the desired image, we have a degradation in the generated image quality. When the remember set is closer to the training set, as in the validation set of CelebA, we see the generation quality remains high.
\input{supp/figs/generate_increase_glasses}
\input{supp/figs/generate_debias_male}

\input{supp/figs/generate_no_training_data}

\section{Results}\label{supp:sec:results}
In this section, we discuss additional results associated with experiments from our \arxiv{}{main} paper. We show:
\begin{enumerate}
    \item Results evaluating our method on the remember set $\mathcal{D}_R$.
    \item An extension of the experiment concerning forgetting an identity, to an unknown identity that is outside the training set.
    \item A full comparison of the experiment for forgetting without any access to the training set.
    \item Evaluation of the effect that training set size has on our method's running time.
    \item An extension of our ablation study, comparing the preservation of the training set distribution.
\end{enumerate}
\input{supp/figs/forget_identity}

We first discuss how our method preserves the NLL distribution of the remember set $\mathcal{D}_R$. In \arxiv{\cref{sec:experiments}}{Sec\onedot4 in the main paper}, we showed results focusing on the forget set $\mathcal{D}_F$. We now show results, focusing on $\mathcal{D}_R$. This is demonstrated by showing this distribution before and after taming, as can be seen in \cref{supp:fig:preserve_dist}. The figure visualizes the differences between the normalized density histogram of the base model ($\theta_B$) and the tamed model ($\theta_T$), along with normal estimation of these distributions, denoted as $\hat{\theta}_B$ and $\hat{\theta}_T$ respectively, on the training set and an unseen set. As we trained $\theta_B$ on the training set of CelebA\cite{celeba}, we used the validation set of CelebA as an unseen set on all upcoming demonstrations, unless specified otherwise.
The distribution pairs in \cref{supp:fig:preserve_dist} are all similar, indicating that we sucessfully forget the target(s), without heavily impacting the rest of the distribution.

\input{supp/figs/forget_without_train_set_full}
Next, we discuss the experiment of forgetting an identity, as shown in \arxiv{\cref{subsec:forget_identity_train_set}}{Sec\onedot4.1 in the main paper}. In \cref{supp:tab:forget_identity_out_of_train} we present the results for an experiment of forgetting an identity outside the training set, using an identity from an unseen data set. We see the results are similar to \arxiv{the ones in \cref{tab:forget_identity}}{the experiment from the main paper} --- we are still able to ``forget'', with similar runtime.
\input{tables/forget_identity_out_of_train/table}

In \cref{supp:fig:forget_without_trainset} we show a full comparison for the experiment on forgetting without the training set, shown in \arxiv{\cref{fig:forget_without_trainset}}{Fig\onedot5 in the main paper}. We see that regarding the NLL distribution of the tamed model $\theta_T$, while there is a degredation, it is still close to the distribution of the base model $\theta_B$ on the same data.

\input{supp/tables/forrget_identity_timing}
Now we turn to inspect whether the time to forget an identity depends on the number of images the model was trained on. To do so, we trained an additional base model just on CelebA. This model was trained on $162{,}770$ images, while the original one, trained additionally on FFHQ\cite{stylegan}, was trained on $232{,}770$. For both models, training stopped with the same performance (in NLL) on CelebA's training set.
In \cref{tab:forget_identity_timing}, we compare the running time of these models, and see that in time that is relative to the model's training time, taming a model that was trained on CelebA only takes longer, but this time is still relatively small, less than 1\% of the total training time of the model.

As discussed in \arxiv{\cref{sec:ablation}}{Sec\onedot5 in the main paper}, in \cref{supp:fig:ablation_distirbution} we compare the distribution of NLL values on the training set of the base model, for different ablated models. Some models are not shown in the figure, as they have a distribution that is visually indistinguishable from the shown distributions of the base and tamed models. The figure shows that without using the forward KL divergence loss ($\cancel{\mathcal{L}_{KL_F}}$), the distribution is worse, but it's also more ``narrow'', fitting the mode-seeking behavior of the reverse KL divergence. On the contrary, without the reverse KL divergence ($\cancel{\mathcal{L}_{KL_R}}$), which is known to be important for generative tasks, the performance is bad, and fits the mean-seeking behavior of forward kl divergence, attempting to cover more regions.
\input{supp/figs/ablation_distribution}
\section{Additional details}\label{supp:sec:additional_details}
\input{supp/figs/baseline_val_distribution}
Next, we elaborate on technical details regarding the implementation of our method, as explained in \arxiv{\cref{sec:experiments}}{Sec\onedot 4 in the main paper}.
We trained a Glow\cite{glow} base model to produce RGB images with dimensions $128\times128$. The training was done for 590K iterations with batch size of 32, for a \mbox{total} of  $316.3$ hours, using 4 12GB Titan Xp GPUs. The model has $4$ blocks of $32$ flows, each consisting of activation normalization layers, $1\times1$ LU decomposed convolution and additive coupling. The model is trained using an Adam \cite{adam} optimizer with learning rate $5\cdot10^{-5}$ and betas $(\beta_1, \beta_2) = (0.9, 0.999)$. Images are qunatized to $5$ bits and learned using the continuous dequantization process as done in previous work \cite{glow,gen_models_eval_dequnatization}. For the forgetting process, we use a threshold of $\delta=4$ and a bound of $\epsilon=0.3$, along with hyperparameters $\alpha=0.6$ and $\gamma=0.6$, in all our experiments.

Now, we discuss the normality assumption as explained in \arxiv{\cref{subsec:task}}{Sec\onedot3.3 in the main paper}. We assumed the NLL distribution of the base model on the training data is normal. To support this assumption, \cref{supp:fig:base_distribution} visually compares the distribution with a normal estimation, along with \mbox{QQ-plots} that further support this claim.
We also performed a \textit{Kolmogorov–Smirnov test} \cite{ks_test} and received a p-value of $0.95,0.54$ on 2000 random samples of CelebA's training and validation sets, respectively. Thus, these results suggest that assuming a normal distribution is
reasonable.

redpajama_set_name: RedPajamaArXiv


Brilliant! Well done. I've had a Fitbit for almost 2 years and it does prompt me to walk more (though the effect has faded a bit from the first flush of enthusiasm). I've not cumulated a year's stats though. Maybe I will for 2017, or maybe i'll take the easier option and start now for 2018!
Very well done, I have loved your walks with smiley Riley.


SCREENSHOT 1. This Tera Term Pro capture lists everything we can do with the MRF24J40 Radio Utility Driver. It looks like this could be fun even without having to put on a pointy star-studded hat.
SCREENSHOT 2. There are some really interesting menu options here, as well. However, our goal is to send and receive some data. So, I figure we won't be configuring any external PAs or LNAs today.
SCREENSHOT 3. As you can see in this capture, the lowest power output we can dial in is between 0.001 and 0.0001 milliwatts. The MRF24J40MA is able to track a signal with a single strength as low as -94 dBm. The MRF24J40MA's receiver overloads at anything above + 5 dBm.
PHOTO 6. If you really want to get serious with Mi Wi or ZigBee, you need one of these.
board with a second module.
I've downloaded the MRF24J40 radio utility driver which we'll use to initially expose the capabilities of the module. The radio utility driver will allow us to configure the MRF24J40MA and run basic transmission and reception tests. The radio utility driver package comes as a collection of MPLAB projects with precompiled hex files. So, we only need to program the PICDEM Z hex file into each of the PICDEM Z/MRF24J40MA board pairs. After I set up a Tera Term Pro session for 19200 bps, eight data bits, no parity, and one Stop bit for each node, we should be ready to rock and roll. According to the utility driver user information, we should be welcomed by a Main Menu by way of the PIC18LF4620's RS-232 port and Tera Term Pro. Behold Screenshot 1.
What moves you in Screenshot 1 Personally, I really want to jump to the Ping Pong Test, but I think we oughta start by punching in "a." Okay, "a" it is. Here we go again. What moves you in Screenshot 2 I like "b." Recall that the MRF24J40MA datasheet specified a typical range of 400 feet. Well, now we know how far 1.0 milliwatts (+0 dBm) off transmit power will take us. I chose to keep the output power at +0 dBm.
I messed around with the other MRF24J40 radio utility driver menu options and finally broke down and ran the Ping Pong Test. As you can see in Screenshot 4, there's nothing new to report. The radio utility driver is definitely aimed at the RF engineers that wear those funny hats. Let's go to Las Vegas.
Mi Wi is a Microchip protocol that can be used in very simple wireless 802.15.4 networks. Our Mi Wi network will consist of a Mi Wi Coordinator and a Mi Wi End Device with both nodes configured as shown in Photo 5. The Mi Wi package is a free download from the Microchip website and comes with ready-to-run code for a PAN Coordinator and End Device. I've loaded up the PICDEM Z hosts. So, let's get the PAN hot.


Over the last few months our impromptu Friday happy hours have taken a decidedly musical turn. A growing group of Substantial staffers began bringing in their drum machines, synths, sequencers, and other bits of gear and closing out the week with a jam session. Playing for hours, the music shifts organically between genres, meandering from abrasive techno to a wandering series of tones. Admittedly, some parts of these jam sessions find their groove more than others. We even took a Saturday night and hosted Gate night, inviting our musical friends to join us in the fun. A sizable fraction of Seattle's analog synth collection made its way to our office and we had a great time with everyone.
For this Substantial Session we've created a "greatest hits" of sorts of the jam sessions to date. All live, all improvised, and all fun for us. If you're interested in joining us for a jam session, let us know. We hope you enjoy the recording as much as we enjoyed making it.
No tracklisting since this is a series of recordings from live jam sessions. Consider everything to be untitled.


text:
Q: Error getting prediction explanation using shap_values when using scikit-learn pipeline? I am building an NLP model to predict language type (C/C++/C#/Python...) for a given code.
Now I need to provide an explanation for my model prediction. For example the following user_input is written in Java and the model is predicting that, but I need to show the users why it predicts so.
I am using shap_values to achieve this.
For some reason, the following code results in an error (I have added the error at the bottom).
Please advise how can I get shap_values and plots for my model predictions.
Link to data: https://sharetext.me/bd68ryvzi0
Code:
import pandas as pd

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import FunctionTransformer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline

# Loading Data:
DATA_PATH = r"sample.csv"

data = pd.read_csv(DATA_PATH, dtype='object')
data = data.convert_dtypes()
data = data.dropna()
data = data.drop_duplicates()

# Train/Test split
X, y = data.content, data.language
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)

# Model params to match:
# 1. Variable and module names, words in a string, keywords: [A-Za-z_]\w*\b
# 2. Operators: [!\#\$%\&\*\+:\-\./<=>\?@\\\^_\|\~]+
# 3. Tabs, spaces and Brackets: [ \t\(\),;\{\}\[\]`"']
# with the following regex:
token_pattern = r"""(\b[A-Za-z_]\w*\b|[!\#\$%\&\*\+:\-\./<=>\?@\\\^_\|\~]+|[ \t\(\),;\{\}\[\]`"'])"""

def preprocess(x):
 """ Clean up single-character variable names or ones constituted of a sequence of the same character """
 return pd.Series(x).replace(r'\b([A-Za-z])\1+\b', '', regex=True)\
 .replace(r'\b[A-Za-z]\b', '', regex=True)

# Pipe steps:
# Define a transformer:
transformer = FunctionTransformer(preprocess)
# Perform TF-IDF vectorization with our token pattern:
vectorizer = TfidfVectorizer(token_pattern=token_pattern, max_features=3000)
# Create Random Forest Classifier:
clf = RandomForestClassifier(n_jobs=4)

pipe_RF = Pipeline([
 ('preprocessing', transformer),
 ('vectorizer', vectorizer),
 ('clf', clf)]
)

# Setting best params (after performing GridSearchCV)
best_params = {
 'clf__criterion': 'gini',
 'clf__max_features': 'sqrt',
 'clf__min_samples_split': 3,
 'clf__n_estimators': 300
}

pipe_RF.set_params(**best_params)

# Fitting
pipe_RF.fit(X_train, y_train)

# Evaluation
print(f'Accuracy: {pipe_RF.score(X_test, y_test)}')

user_input = [""" public class Fibonacci {

public static void main(String[] args) {

int n = 10;

System.out.println(fib(n));

}

public static int fib(int n) {

if (n <= 1) {

return n;

}

return fib(n - 1) + fib(n - 2);

}

} """]

import shap

shap.initjs()
explainer = shap.TreeExplainer(pipe_RF.named_steps['clf'])
observation = pipe_RF[:-1].transform(user_input).toarray()
shap_values = explainer.shap_values(observation)

Load the data and run it to get the following error:

ExplainerError: Additivity check failed in TreeExplainer! Please
ensure the data matrix you passed to the explainer is the same shape
that the model was trained on. If your data shape is correct then
please report this on GitHub. Consider retrying with the
feature_perturbation='interventional' option. This check failed
because for one of the samples the sum of the SHAP values was
46609069202029743624438153216.000000, while the model output was 0.004444. If this difference is acceptable you can set check_additivity=False to disable this check.

A: I have figured out how to fix it, posting to help others :)
import pandas as pd

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.preprocessing import FunctionTransformer
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.pipeline import Pipeline

import re
from lime.lime_text import LimeTextExplainer

from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"

# Loading GitHub Repos data containing code and comments from 2.8 million GitHub repositories:
DATA_PATH = r"/Users/stevesolun/Steves_Files/Data/github_repos_data.csv"

data = pd.read_csv(DATA_PATH, dtype='object')
data = data.convert_dtypes()
data = data.dropna()
data = data.drop_duplicates()

# Train/Test split
X, y = data.content, data.language
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y)

# Model params to match:
# 1. Variable and module names, words in a string, keywords: [A-Za-z_]\w*\b
# 2. Operators: [!\#\$%\&\*\+:\-\./<=>\?@\\\^_\|\~]+
# 3. Tabs, spaces and Brackets: [ \t\(\),;\{\}\[\]`"']
# with the following regex:
token_pattern = r"""(\b[A-Za-z_]\w*\b|[!\#\$%\&\*\+:\-\./<=>\?@\\\^_\|\~]+|[ \t\(\),;\{\}\[\]`"'])"""

def preprocess(x):
    """ Clean up single-character variable names or ones constituted of a sequence of the same character """
    return pd.Series(x).replace(r'\b([A-Za-z])\1+\b', '', regex=True)\
        .replace(r'\b[A-Za-z]\b', '', regex=True)

# Pipe steps:
# Define a transformer:
transformer = FunctionTransformer(preprocess)
# Perform TF-IDF vectorization with our token pattern:
vectorizer = TfidfVectorizer(token_pattern=token_pattern, max_features=1500)
# Create Random Forest Classifier:
clf = RandomForestClassifier(n_jobs=-1)

pipe_RF = Pipeline([
     ('preprocessing', transformer),
     ('vectorizer', vectorizer)]
    )

# Setting best params (after performing GridSearchCV)
best_params = {
    'criterion': 'gini',
    'max_features': 'sqrt',
    'min_samples_split': 3,
    'n_estimators': 300
}

clf.set_params(**best_params)
# Here I am preprocessing the data:
X_train = pipe_RF.fit_transform(X_train).toarray()
X_test = pipe_RF.transform(X_test).toarray()

# Fitting the model outside the pipe - feel free to show if possible to do it inside the pipe + fit_transform the train and test sets.
clf.fit(X_train, y_train)

# Evaluation
print(f'Accuracy: {clf.score(X_test, y_test)}')

user_input = """ def fib(n):
                    a,b = 0,1
                    while a < n:
                        print(a, end=' ')
                    a,b = b, a+b
                    print()
                    fib(1000)

   """
clf.predict(pipe_RF.transform(user_input))[0]

prediction = clf.predict(pipe_RF.transform(user_input))[0]
predicted_class_idx = list(clf.classes_).index(prediction)

import shap

shap.initjs()
explainer = shap.TreeExplainer(clf, X_train)
observation = pipe_RF.transform(user_input).toarray()
shap_values = explainer.shap_values(observation)

shap.force_plot(explainer.expected_value[predicted_class_idx], shap_values[predicted_class_idx], feature_names=vectorizer.get_feature_names_out())

redpajama_set_name: RedPajamaStackExchange


The paparazzi always had a thing for Princess Diana. And why not? She was young, beautiful, and charismatic, even as a shy newlywed. So when she and Prince Charles flew to the Bahamas for a 10-day vacation in February 1982, they tried to fool the British press by traveling as Mr. and Mrs. Hardy. That trick didn't work at all!
The tabloids struck gold. Diana was about five months pregnant at the time with Prince William (who was born in June that year). But instead of hiding her bump beneath a maternity swimsuit, she wore a bikini. When the photos appeared along with headlines like "Bahama Mama," Queen Elizabeth called it "the blackest day in the history of British journalism."
Sure, we wonder what the fuss was about now. But what strikes us about these photos are how happy the two parents-to-be look, and how affectionate they were with each other. There 10-day stay in the sun was a true babymoon, and these photos prove it! Swipe through and see for yourself. One in particular is kind of jaw-dropping, as we'd NEVER seen Charles and Diana that loosey-goosey before.
This photo seems tame now, but back in 1982, it made headlines all over the world!


Want the inside scoop for the University of Sudbury? Check out the U Sudbury podcast series! Our guests share their expertise and experiences on student life, campus services, local opportunities, university programs and more - to help you make the most of your university experience! Check out this link to listen: https://soundcloud.com/usudburypodcasts/tracks (https://soundcloud.com/usudburypodcasts/tracks) or click on the podcast section to the right - it will bring you to our Soundcloud page where you will see all 12+ podcasts to date! Follow us on Soundcloud to hear our new podcasts as soon as they're available! Thanks for listening, and let us know what you think!
We would be pleased to have you join us!
In this rapidly changing field, employers seek out our graduates because they are familiar with the latest technologies and the innovative approaches required in today's world.
provides a better understanding of Francophone culture, customs, traditions, legends and beliefs that have shaped it.
promotes an understanding of Indigenous peoples, their traditions, aspirations and participation in local, national and international communities.
will give you the tools to critically examine your own worldviews, as well as those of others; will allow you to explore various questions and areas of personal interest; and will teach you how to think about issues that touch your life.
approaches the study of religion as a social phenomenon, focusing on the relationship between religion and culture, recognizing that religions have occupied a very central place in virtually all civilizations.


Lakewood is a great community near Dallas in Texas. As a part of the "golden triangle" of the Dallas area, you will find a lot of gold for sale. You can buy things like gold chains, gold teeth, gold spinning rims, and a big golden cup. If you are near Lakewood and want to grab some precious metal, whether that be gold or silver or even platinum, stop in some of these wonderful places or give us a call at Gramercy Gold.


Studio apartment Kandija 1 is located in the beautiful bay of Kandija on Lošinj. In the bay there are about 15 houses and is a great choice for a peaceful and relaxing holiday. In the apartment there is all the necessary, block kitchen, double bed, bathroom, and in front of the terrace with barbecue. To the sea and beach there are only 60 meters.


text:
Marilyn Manson Hospitalized After Two Giant Stage Props Fall On Him
Marilyn Manson was rushed to a hospital in New York City last night (Sept. 30) after a large stage prop fell on him while he was playing a show.
The accident took place at the Hammersmith Ballroom in Manhattan, about 45 minutes into his performance. The video above shows him kneeling down in front of two giant pistols attached to a frame, which topples forward over him while he's singing "Sweet Dreams (Are Made of This)." Crewmen then rush onto the stage and manhandle the frame back into position, while others lean over Manson, who remains on the floor.
"It was so fast, at first I thought it was part of the show," fan Ciro Castro tole the New York Daily News. "But then I saw the guitarist's face, and the faces of some of the others, and they looked really scared. I was like, 'Oh, this is not a joke.' He moved for a minute at first but then he was still. They turned off the lights but we couldn't see they were trying to cover him with a blanket."
Manson lay there for at least 10 minutes before emergency services lifted him onto a stretcher. A spokesman later reported: "Manson suffered an injury towards the end of an incredible NYC show. He is being treated at a local hospital." While no details were revealed, another spokesperson said the singer would be "okay."
Manson apparently also injured himself during a show the previous night (Friday, Sept. 29) in Pittsburgh, PA. He jumped off the stage while performing "Beautiful People" and on returning to the platform told the audience: "I just broke my ankle but it's okay." He releases his latest album Heaven Upside Down on Oct. 6.
Rock's Worst Concert Mishaps and Tragedies
Next: Marilyn Manson for Vice President?
Filed Under: Marilyn Manson


text:
Symantec takes a step into the future with AI powered cyber defence
The pace and sophistication of new threats facing organisations is daunting. That's according to Symantec, who in 2015, reported more than 430 million new pieces of malware.
To combat these threats, the cyber security organisation has unveiled Symantec Endpoint Protection 14, what it positions as the next evolution in endpoint innovation.
Powered by artificial intelligence, Symantec Endpoint Protection 14 is the industry's first solution to marry essential endpoint technologies with advanced machine learning.
Symantec now protects 175 million consumer and enterprise endpoints, 163 million email users, 80 million web proxy users, and processes nearly eight billion security requests across these products every day.
Mike Fey, president and chief operating officer at Symantec, says the new endpoint protection is a major leap forward in endpoint protection.
"Multi-layered protection, enabled by artificial intelligence, backed by the world's largest and most powerful threat intelligence force, and powered by the cloud - this is literally the smartest choice in endpoint technologies," he explains.
"Symantec Endpoint Protection 14 is an essential element of an integrated cyber defense strategy that enterprises require to combat today's advanced threats."
Sharon Finney, corporate data security officer at Adventist Health System, says that as a healthcare institution, they're dealing with sensitive patient records and strict regulatory standards, so using innovative technology protections is critical to their business.
"However, as a large institution with tens of thousands of users, we do not take unnecessary risks by immediately deploying every new technology or point product, and count on vendors like Symantec to integrate those innovations."
Remote working trend bolsters cybersecurity investment - but downturn predicted
Cisco, ESET, Fortinet crowned in Canalys Cybersecurity Leadership Matrix, while Symantec, IBM and McAfee underperform
HCL takes over Broadcom's Symantec security consulting
Accenture named Leader in managed security MarketScape
All the winners from Microsoft's 20/20 security awards
Kaspersky named one of the top 100 global innovators
IronNet expands Asia Pacific presence with new strategic partnership
"The combination of M.Tech's extensive network in Asia Pacific and our unparalleled expertise in threat intelligence and detection will help more enterprises across the region to proactively identify and take down known and unknown threats before they happen."More
PDI acquires Cybera and ControlScan MSS to protect against security threats
The acquisition complements PDI's existing industry-focused cloud product strategy, bringing customers a fully managed, cloud-based network security solution, the company states.More


text:
27 September 2018 - Investing
Trilogy set to acquire Milton office space for new trust
We're excited to announce that we've entered terms to acquire 16 Marie Street, Milton in Brisbane's inner-city suburban commercial office market.
The six-storey, multi-tenant commercial property boasts almost 4,000 sqm of net lettable area and 33 on-site parking bays. Expected to settle in early December, the property will serve as the single asset in our recently established Milton Office Trust, which is officially open for investment.
Our Managing Director Philip Ryan said the attractive, modern building exemplifies key qualities and features that would hold genuine appeal to potential investors considering single-asset property trusts.
"The building is less than 10 years old. The income is anchored with a lease of two floors to long term tenant, SuperConcepts – who are a subsidiary of AMP" he said.
"Milton Office Trust comes to market offering an attractive yield derived from a diversified list of six tenants from different industries.
"This is an excellent opportunity for direct investors and self-managed super funds seeking yield from exposure to the property market. The Milton Office Trust will provide investors at various stages of their investment journey the opportunity to experience the returns available from commercial property that they might otherwise be unable to afford."
Milton is a popular commercial precinct, defined by its easy access to the CBD, casual riverside living, and a thriving entertainment precinct. It is filled with bustling restaurants, cafés and boutique retailers.
"Since the gentrification of South Brisbane and more recently, Newstead, New Farm and Teneriffe, many small-to-mid sized businesses have been drawn to those areas. Milton has appeal to providers of professional services particularly in the infrastructure, building and mining industries…"
"Milton's residential population is also growing and we're starting to see the emergence of fresh residential development in the area.
"This brings significant upside to our investment. More people living, working and enjoying all Milton has to offer enhances the suburb's appeal to the business sector.
"Businesses seeking lifestyle balance for their employees, access to amenity and proximity to Brisbane's CBD are beginning to see Milton for its true worth."
Philip forecasts that the Trust will provide investors with a monthly distribution yield equivalent to 7.50% p.a. over the first 18 months of the six-year investment term.
The material on this website is intended only to provide a summary and general overview on matters of interest. Trilogy is only licensed to provide general financial product advice on its own products and does not consider your objectives, financial situation or needs when providing any information or advice. You should consider whether the advice is suitable for you and your personal circumstances and we recommend that you seek personal financial product advice on your objectives, financial situation or needs and obtain and read the relevant product disclosure statement before making any investment decision.
Trilogy has issued a Product Disclosure Statement for the Milton Office Trust dated 24 September 2018 which is available at www.trilogyfunds.com.au or by contacting us. Applications will only be accepted on the current application form that accompanies the PDS. You should obtain a copy, understand the risks, and seek personal advice from a licensed Financial Adviser before investing. Investment in the Trust is subject to terms and conditions, and risks which are disclosed in the PDS. These risks include the risk of losing income or principal invested. The Trust is not a bank deposit and Trilogy does not guarantee its performance. Trilogy provides only general financial product advice on its own products and does not consider your objectives.
Cash holdings a drag on investment performance
5 New Year's resolutions for investors
Australian house prices on the rise again


The arrangement of unorganized data into a structured digital format is our core competency and we are extremely sincere about it. We hold great proficiency in data entry and data processing services and we are perfect for the job in terms of expertise, experience, infrastructure, and technology.
Data Processing – What does it stand for?
Data Processing is a comprehensive term and includes everything to do with turning raw, unorganized data into an organized digital format. The goal of data processing services is to streamline the primary and secondary data gathered in various formats from marketing surveys, feedback forms, registration, subscription forms etc., into tabular, textual or other such formats, so that the researched data could be used by enterprises to ascertain business trends, patterns, or projections for future risk management; or to acquire crucial insight into the demographics for decision making.
Capturing raw data out of all-inclusive input media including surveys, feedback forms, scanned files, PDF documents, images, web pages etc.
Organizing and converting captured data into the desired digital formats.
Putting logical reasoning and analytical faculties to use to evaluate data and reach an inference. The data analysis methods our team employs include data visualization, data mining, business intelligence, and text analytics.
Presenting the processed data into a visually comprehensible and useful tabular or textual format, as per the specific requirements of clients is something we do almost every day.
We are a team of experienced and self-motivated professionals. We have many years of experience behind us in dealing with large volumes of diverse data for processing and eventual digital transformation.
At DataEntry4OS.com, each member of our team has been carefully recruited to ensure that only qualified and well-trained staff engages in our projects. Each one of us is well-versed in the English language, has fast typing speed with optimum accuracy and we regularly attend in-house orientation and training sessions before beginning to work on the undertaken projects.
Besides having an excellent team, we have a host of other favorable conditions as an offshore outsourcing concern, such as strategic location, qualified English-speaking manpower, state-of-the-art technology, robust infrastructure, and idealistic zeal for work.
Our diverse clientele outsources a crucial part of their business to us without a flinch of an eye, confident of the excellent quality services in the least turnaround time possible and at very competitive rates.
No data job is too complicated for us. We, therefore, invite any queries/ clarifications that you might still have regarding our data processing services. Reach us at info@dataentry4os.com and we assure we'll get back to you soon.


Please read the ESLstarter Referral Program Terms and Conditions carefully.
Promotional Materials. ESLstarter shall make available to Affiliate certain text links, and/or other graphic or textual material for display and use on the Affiliate website or social media (the "Promotional Materials"). Affiliate shall display the Promotional Materials on Affiliate's website or social media prominently and as Affiliate sees fit, provided that the manner of display shall be subject to the terms and conditions of this Agreement.
(a) Affiliate may not use any graphic, textual or other materials to promote ESLstarter's website, products or services other than the Promotional Materials, unless ESLstarter agrees to such other materials in writing prior to their display.
(b) Affiliate may only use the Promotional Materials for the purpose of promoting ESLstarter's website (and the products and services available thereon), and for linking to the ESLstarter website.
(c) Affiliate will not alter, add to, subtract from, or otherwise modify the Promotional Materials as they are prepared by ESLstarter. If Affiliate wishes to alter or otherwise modify the Promotional Materials, Affiliate must obtain prior written consent from ESLstarter for such alteration of modification.
(d) The Promotional Materials will be used to link only to ESLstarter's website, to the specific page and address as specified by ESLstarter.
(e) The Affiliate must use their unique URL in order to gain successful referral credits. Only referrals submitted via the Affiliate's unique URL will be logged in the system and thus be considered as "valid referrals" (subject also to the referred candidate successfully taking up a teaching post through ESLstarter). Restated – referrals who do not enter the system via the Affiliate's unique URL will not be considered as "referred candidates" and therefore no credit will be due in this case.
License. ESLstarter hereby grants to Affiliate a nonexclusive, nontransferable license (the "License") to use the Promotional Materials as specified under the terms and conditions of this Agreement. The term of the License shall expire upon the expiration or termination of this Agreement.
Intellectual Property. ESLstarter retains all right, ownership, and interest in the Promotional Materials, and in any copyright, trademark, or other intellectual property in the Promotional Materials. Nothing in this Agreement shall be construed to grant Affiliate any rights, ownership or interest in the Promotional Materials, or in the underlying intellectual property, other than the rights to use the Promotional Materials granted under the License, as set forth in Section 3.
Relationship of Parties. This Agreement shall not be construed to create any employment relationship, agency relationship, or partnership between ESLstarter and Affiliate. Affiliate shall provide services for ESLstarter as an independent contractor. Affiliate shall have no authority to bind ESLstarter into any agreement, nor shall Affiliate be considered to be an agent of ESLstarter in any respect.
(a) In exchange for Affiliate's display of the Promotional Materials, and for Affiliate's compliance with and performance of the terms and conditions of this Agreement, ESLstarter shall pay to Affiliate a commission (the "Commission") in the amount of £75 per introduction and successful recruitment of a candidate referred to the ESLstarter website through a link on the Affiliate's website or social media.
(b) This amount is subject to change by ESLstarter at any time. Notification to Affiliate of any change in commission percentage will be given by ESLstarter at the email address on hand for the Affiliate.
(c) The "Commission" is triggered when a candidate referred by the Affiliate is successfully placed onto one of the ESLstarter teaching programs and after said candidate arrives in host country and has completed 30 days of employment.
(d) ESLstarter shall keep accurate and up-to-date records of the data used to determine the total amount of Commissions owed to Affiliate. Affiliate shall be given reasonable access to these records upon request, and is available through the affiliate area on the member page at www.eslstarter.com/affiliates/affiliate-area/. Any discrepancy between the amount of Commissions owed according to these records, and the actual amount of Commissions paid to Affiliate in any period or periods shall be rectified by ESLstarter within 14 days of discovering such discrepancy.
(e) ESLstarter shall pay all Commissions accrued and payable to Affiliate within 7 days of the first day of each month (the "Commission Payment Date"). Payment is made via the Affiliate PayPal account. ESLstarter shall pay all Commissions via Paypal and not through any other payment method.
(f) In the event that Affiliate materially breaches this Agreement and ESLstarter terminates this Agreement within 30 days of such breach, then any accrued and payable Commissions owing to Affiliate shall be forfeited, and ESLstarter shall not be obligated to pay such Commissions to Affiliate.
(a) Affiliate has the legal authority to enter into this Agreement and to be bound to the promises, covenants, and other duties set forth in this Agreement.
(c) Affiliate has obtained any necessary clearances, licenses, or other permission for any intellectual property used on Affiliate's website. Nothing on Affiliate's website infringes upon the intellectual property rights of any person or entity. No person or entity has brought or threatened an action claiming such infringement, nor does Affiliate have any reason to believe that any person or entity will bring or threaten such a claim in the future.
(d) Affiliate will not use the Promotional Materials in any manner other than those set forth in Section 2 above.
(e) Affiliate will not make any claim to ownership of the Promotional Materials, or of the copyright, trademark, or other intellectual property therein.
(f) Affiliate will not publish or otherwise distribute any advertising materials for Affiliate's website that reference ESLstarter or ESLstarter's website unless ESLstarter gives prior written consent to the distribution of such materials. Affiliate will not use ESLstarter's name (or any name that is confusingly similar to ESLstarter's name) for any purpose on its website, in its promotional materials, or in any other context except to promote ESLstarter's website as specified in this Agreement. Affiliate will not register any domain name that incorporates ESLstarter's name, or that is confusingly similar to ESLstarter's name.
(g) Affiliate will not engage in the distribution of any unsolicited bulk emails (spam) in any way mentioning or referencing ESLstarter or ESLstarter's website.
Indemnification. Affiliate shall indemnify ESLstarter and hold harmless ESLstarter from any claim, damage, lawsuit, action, complaint, or other costs arising out of any breach of Affiliate's warranties set forth in Section 7 above. Affiliate shall also indemnify and hold harmless ESLstarter for any damage, loss or other cost arising out of the use or misuse by Affiliate of the Promotional Materials.
Confidentiality. Any information that Affiliate is exposed to by virtue of its relationship with ESLstarter under this Agreement, which information is not available to the general public, shall be considered to be "Confidential Company Information." Affiliate may not disclose any Confidential ESLstarter Information to any person or entity, except where compelled by law, unless Affiliate obtains prior written consent for such disclosure from ESLstarter.
(a) This Agreement shall take effect immediately, and shall remain in full force and effect indefinitely, or until terminated pursuant to this Section 10.
(b) Either Party shall have the right to terminate this Agreement at any time and for any cause. The terminating Party must give written notice to the other Party at least 30 days prior to the intended date of termination.
Taxes. ESLstarter shall not be responsible for any taxes owed by Affiliate arising out of Affiliate's relationship with ESLstarter as set forth in this Agreement. ESLstarter shall not withhold any taxes from the Commissions paid to Affiliate.
Limitation of Liability. ESLstarter shall not be liable for any loss of profits or costs, or for any direct, indirect, special, incidental or consequential damages, including costs associated with the procurement of substitute goods or services (whether ESLstarter was or should have been aware or advised of the possibility of such damage), arising out of or associated with any loss, suspension or interruption of service, termination of this Agreement, use or misuse of the Promotional Materials, or other performance of services under this Agreement.
Entire Agreement. This Agreement constitutes the entire agreement between ESLstarter and Affiliate, and supersedes any prior understanding or representation of any kind preceding the date of this Agreement. There are no other promises, conditions, understandings or other agreements, whether oral or written, relating to the subject matter of this Agreement.


text:
Weisenfeld
Avid communicator and natural-born marketer with a passion for, well, everything.
Where It All Started
Since a very young age, Craig has always loved to be around and communicate with people. He believes that sharing thoughts, feelings, ideas, passions, and time with others is the key to life.
In the Early Days
Craig studied Broadcast Journalism at the University of Maryland, where he combined his passions for writing and visual media. During the summers, though, he still made time to return to work at the overnight summer camp on Cape Cod where he spent his summers from elementary school through college.
Beginning in senior year, Craig began to transition from traditional journalism to the world of content and digital marketing. After spending time as a freelance video editor, he became a one-man digital marketing team at two youth programming organizations, most recently with the New York Red Bulls of Major League Soccer.
What He Brings to the Table
Craig has spent his career using digital marketing to connect parents with programs and activities for their children. He has a wealth of experience planning and executing multi-channel campaigns with paid social media, organic social media, paid search, email marketing, website management, and more.
What Gets Him Up in the Morning
FTK – For The Kids. Craig is a strong believer that the next generation will make the world a better place and knows education plays a huge role in helping make that happen.
In His Spare Time
Craig is based in Brooklyn, NY, where he likes to do fun things every day of the week: go to concerts, go to the movies, play soccer, or just hang with friends.