text:
Science 15 Mar 2017 A perfect storm of f ...
A perfect storm of fire and ice may have led to snowball Earth
Published Mar 15, 2017, 12:26 pm IST
Updated Mar 15, 2017, 12:27 pm IST
What caused the largest glaciation event in Earth's history, known as 'snowball Earth'?
A trail of smoke can be seen from the 2013 eruption of the Pavlof volcano, which spewed an ash cloud 20,000 feet into the atmosphere that traveled southeastward over the North Pacific Ocean. The volcanos that may have contributed to snowball Earth spanned almost 2000 miles and erupted continuously for years. (Image courtesy of NASA)
Geologists and climate scientists have been searching for the answer for years but the root cause of the phenomenon remains elusive.
Now, Harvard University researchers have a new hypothesis about what caused the runaway glaciation that covered the Earth pole-to-pole in ice.
The research is published in Geophysical Research Letters.
Researchers have pinpointed the start of what's known as the Sturtian snowball Earth event to about 717 million years ago — give or take a few 100,000 years. At around that time, a huge volcanic event devastated an area from present day Alaska to Greenland.
Harvard professors Francis Macdonald and Robin Wordsworth thought not.
"We know that volcanic activity can have a major effect on the environment, so the big question was, how are these two events related," said Macdonald, the John L. Loeb Associate Professor of the Natural Sciences.
At first, Macdonald's team thought basaltic rock — which breaks down into magnesium and calcium — interacted with CO2 in the atmosphere and caused cooling. However, if that were the case, cooling would have happened over millions of years and radio-isotopic dating from volcanic rocks in Arctic Canada suggest a far more precise coincidence with cooling.
Macdonald turned to Wordsworth, who models climates of non-Earth planets, and asked: could aerosols emitted from these volcanos have rapidly cooled Earth?
The answer: yes, under the right conditions.
"It is not unique to have large volcanic provinces erupting," said Wordsworth, assistant professor of Environmental Science and Engineering at the Harvard John A. Paulson School of Engineering and Applied Science. "These types of eruptions have happened over and over again throughout geological time but they're not always associated with cooling events. So, the question is, what made this event different?"
Geological and chemical studies of this region, known as the Franklin large igneous province, showed that volcanic rocks erupted through sulfur-rich sediments, which would have been pushed into the atmosphere during eruption as sulfur dioxide. When sulfur dioxide gets into the upper layers of the atmosphere, it's very good at blocking solar radiation. The 1991 eruption of Mount Pinatubo in the Philippines, which shot about 10 million metric tons of sulfur into the air, reduced global temperatures about 1 degree Fahrenheit for a year.
Sulfur dioxide is most effective at blocking solar radiation if it gets past the tropopause, the boundary separating the troposphere and stratosphere. If it reaches this height, it's less likely to be brought back down to earth in precipitation or mixed with other particles, extending its presence in the atmosphere from about a week to about a year.
The height of the tropopause barrier all depends on the background climate of the planet — the cooler the planet, the lower the tropopause.
"In periods of Earth's history when it was very warm, volcanic cooling would not have been very important because the Earth would have been shielded by this warm, high tropopause," said Wordsworth. "In cooler conditions, Earth becomes uniquely vulnerable to having these kinds of volcanic perturbations to climate."
"What our models have shown is that context and background really matters," said Macdonald.
Another important aspect is where the sulfur dioxide plumes reach the stratosphere. Due to continental drift, 717 million years ago, the Franklin large igneous province where these eruptions took place was situated near the equator, the entry point for most of the solar radiation that keeps the Earth warm.
So, an effective light-reflecting gas entered the atmosphere at just the right location and height to cause cooling. But another element was needed to form the perfect storm scenario. After all, the Pinatubo eruption had similar qualities but its cooling effect only lasted about a year.
The eruptions throwing sulfur into the air 717 million years ago weren't one-off explosions of single volcanoes like Pinatubo. The volcanoes in question spanned almost 2,000 miles across Canada and Greenland. Instead of singularly explosive eruptions, these volcanoes can erupt more continuously like those in Hawaii and Iceland today. The researchers demonstrated that a decade or so of continual eruptions from this type of volcanoes could have poured enough aerosols into the atmosphere to rapidly destabilize the climate.
"Cooling from aerosols doesn't have to freeze the whole planet; it just has to drive the ice to a critical latitude. Then the ice does the rest," said Macdonald.
The more ice, the more sunlight is reflected and the cooler the planet becomes. Once the ice reaches latitudes around present-day California, the positive feedback loop takes over and the runaway snowball effect is pretty much unstoppable.
"It's easy to think of climate as this immense system that is very difficult to change and in many ways that's true. But there have been very dramatic changes in the past and there's every possibility that as sudden of a change could happen in the future as well," said Wordsworth.
Understanding how these dramatic changes occur could help researchers better understand how extinctions occurred, how proposed geoengineering approaches may impact climate and how climates change on other planets.
"This research shows that we need to get away from a simple paradigm of exoplanets, just thinking about stable equilibrium conditions and habitable zones," said Wordsworth. "We know that Earth is a dynamic and active place that has had sharp transitions. There is every reason to believe that rapid climate transitions of this type are the norm on planets, rather than the exception."
Click on Deccan Chronicle Technology and Science for the latest news and reviews. Follow us on Facebook, Twitter.
Tags: moon, earth, exoplanets, solar system
Latest From Science
Doctors confirm false alarm over Indian teacher's ailment in China
Favouritism, regionalism did not spare even Nobel laureate C V Raman
Climate change alters body clock of snakes
World's oceans were warmest ever in 2019: Study
More From Science
One billion animals killed or imperilled by Australian fires, says expert
Dr George Schaller: Make money from tigers and give it to the forests
Map of Milky Way Galaxy reveals presence of stellar nurseries wave
Astronomers measure expansion rate of Universe
Study suggests 'goldilocks stars' are best places to look for life


$5 per family. 2nd Fridays. Regular activities include: Gymnasium Fun - bouncy house, sports, fun games; family Fitness Classes and Fit2Go Van; classes, movies & themes change monthly; drop-In Art and Pottery Projects; game Room Fun - pool, ping pong, foosball, air hockey; child Care Room - drop off young kids for playtime.


I will readily admit that I poke fun at Democratic Rep. Alexandria Ocasio-Cortez as much, if not significantly more so, than the next person.
To be fair, it's hard to blame me.
Ocasio-Cortez willingly provides endless fodder for others to skewer her over.
A ridiculously expensive green tax that would barely move the needle on the thermometer? She's for it.
Childlike petulance when she gets fact-checked? Sounds like her.
Perpetuating a completely fabricated outrage to play the victim card? She's mastered that.
Point being, there is no shortage of things to poke fun at her for.
But one thing we cannot lose amid that lambasting is the fact that, due to her elected position, Ocasio-Cortez can represent a very real danger to this democracy. After all, it's all fun and games until she actually gets her way.
Hollywood actor and noted conservative James Woods made sure to remind us all of this very important fact on his Twitter account.
Scarily enough, Woods might be right. For as much fun as I've had at her expense, it can't be understated that Ocasio-Cortez's rhetoric and potential impact on America is no laughing matter.
First and foremost, Ocasio-Cortez identifies herself as a Democratic Socialist a la Bernie Sanders. Socialism is a scourge on the capitalist foundations this country was built on.
There is something distinctly American about picking yourself up by your own bootstraps, going to work and earning your living. By that same token, there is something disgustingly un-American about government handouts based on the hard work of others, which is what socialism boils down to.
Secondly, as ridiculous as Ocasio-Cortez's rhetoric is, even I must painfully admit that she has a bit of a cult of personality.
The problem is that her followers are utterly unhinged and blindly follow her no matter what cliff she's leading them off.
Thirdly, and bear with me here, Ocasio-Cortez's platform has a hint of President Donald Trump's. Both are a complete disruption to the status quo.
Whereas Trump was a disruption to the lazy and dynastic Republican party of yore, Ocasio-Cortez is a disruption to more centered leftism of Democrats.
The problem is that while Trump is working to minimize government and put as much money in the pockets of Americans, Ocasio-Cortez is working for the complete opposite of that.
Do you think Woods is right about Ocasio-Cortez?
We're all experiencing the impact and effects of what Trump is doing. Imagine if that same level of impact was on the other foot. That's abjectly terrifying.
So yes, continue to denigrate Ocasio-Cortez's lunacy because she deserves it. Just keep in mind that she presents a real danger to this country.
Conservative Tribune reached out to Ocasio-Cortez's team for comment on this story. We did not receive a response at the time of publication.


Longtime Ernie Ball legend Eric Clapton's new album "I Still Do" is officially coming out this Friday, May 20th. The album is produced by Glyn Johns, who worked with Clapton on his 70's classics, Slowhand and Backless.
In anticipation, Eric has released a new animated video for the single, "Spiral." Many of the album's songs are covers, but "Spiral" is a completely new track written for the album, written with two of longtime collaborators, Simon Climie and Andy Fairweather Low.
The video chronicles Clapton's rich career and features a portrait by Peter Blake, the artist responsible for the Beatles' iconic Sgt. Pepper's Lonely Hearts Club Band album cover.
Check out the July 2016 issue of Guitar World, (available this week) to read the full interview with Clapton about the album, which you can now pre-order on iTunes.
Stream "Catch the Blues," another single from the album, via Spotify.


text:
Q: Where to find icons used in Windows 10 Calculator? Does anybody know where to find the icons used in the Win 10 Calculator app?  I know that some of them are in Segoe MDL2 Assets (such as the Calendar icon), but others I can't seem to find at all.  The icons I'm referring to are the ones in the NavigationMenu, such as those next to Scientific, Currency, Volume, etc.  I've included an image below of the Calculator menu for reference.
I know NavigationViewItem has an Icon property, and that property can hold any Icon type (FontIcon, SymbolIcon, etc), I just can't figure out what they used to display these icons.
XAML Options:
<!--Normal Icon-->
<NavigationViewItem Content="Standard" Icon="Calculator" />

<!--FontIcon-->
<NavigationViewItem Content="Volume">
    <NavigationViewItem.Icon>
        <FontIcon Glyph="" FontFamily="Segoe MDL2 Assets" />
    </NavigationViewItem.Icon>
</NavigationViewItem>

Image:

A: MS calculator now is open source, you could get all source code and relevant resource from github: https://github.com/Microsoft/calculator

A: All the icons are in here: https://github.com/Microsoft/calculator/blob/master/src/Calculator/Assets/CalcMDL2.ttf
If you're running the October 2018 update or over, drag the font into an app like https://www.microsoft.com/en-gb/p/character-map-uwp/9wzdncrdxf41?activetab=pivot:overviewtab to preview the characters and get the correct XAML needed to display it.
(Segoe MDL2 Assets font is actually just a subset of common MDL2 Assets, and many Microsoft apps have their own variants. With a bit of smart searching, you can also find the not-public "Full MDL2 Assets" font that has ALL the icons in it.)

redpajama_set_name: RedPajamaStackExchange


text:
Florence King (January 5, 1936 – January 6, 2016) was an American novelist, essayist and columnist. She was the author of Confessions of a Failed Southern Lady.
Latest by Florence King in Chronicles
Results: 7 Articles found.
Anglo-Americana
In 1858, as British and French forces pushed their way to Peking in the Opium Wars, Josiah Tatnall, commander of the neutral American naval squadron, intervened to save the British ships from Chinese guns and tow them safely out of range.
A Song in My Heart, A Hole in My Head
Eleanor Roosevelt and I go way back. My father taught me to read from a stack of her "My Day" columns in 1940. We happened to have a plentiful supply of "My Day" in the house because the doctor had refused to be responsible for my reactionary grandmother's blood pressure unless she stopped reading it.
Moi, le Déluge
"He was just five years old when Mattie Barry, seeking a fresh start in life, moved north with her son and two older daughters to Memphis. . . . Her husband had been killed a year earlier in Itta Bena.
The Goading of America
Fisher Ames is the Founding Father who draws a blank. Few people today have heard of him, yet he wrote the final version of the First Amendment, and his speech on Jay's Treaty, . . . was called the finest example of American oratory by Daniel Webster and Abraham Lincoln.
Remember Kate Millett? She made the cover of Time in 1970 after her dissection of literary machismo, Sexual Politics, became a blockbuster best-seller and won her the title of leading feminist spokesperson.

redpajama_set_name: RedPajamaCommonCrawl


text:
Salamanca Club de Fútbol UDS, previously known as CF Salmantino, is a Spanish football team based in Salamanca, in the autonomous community of Castile and León. Founded in 2013 after the dissolution of UD Salamanca, it currently plays in Segunda División RFEF – Group 1, holding home games at the Estadio Helmántico with a 17,341-seat capacity.

History

After the dissolution of UD Salamanca in 2013, some managers of the entity decided to refound the farm team to preserve the legacy of the historical club. With that aim, they created CD CF Salmantino. At first, it started playing in the same division as the old reserve team of Salamanca. Nevertheless, the Royal Spanish Football Federation decided in 2015 to relegate all the teams of the club to the lowest tier, as they considered the new CF Salmantino to be a newly-founded club, completely independent of UDS.

In the 2015–16 season, the first after the administrative relegation, the club was promoted to fifth tier, and repeated success for coming back to Tercera División two years after the judicial statement.

On 24 June 2018, just in their first season after the comeback, Salmantino promoted to Segunda División B by eliminating Compostela in the last round of the promotion play-offs. On 4 July 2018 José Miguel Campos was appointed the new head coach of the club.  On 24 July, the club requested the Royal Spanish Football Federation to change the name to Salamanca CF UDS, recovering the name of former side UD Salamanca, and to use their logo. However, this change was initially not admitted, arguing that the request was presented out of date, but was finally approved on 14 September 2018.

On 18 September 2019 it was announced that Pablo Cortes would be the coach for the entire 2019-20 season. But on 28 October 2019 the club has fired him.

Club background
UD Salamanca (1923–2013)
Club naming
CD CF Salmantino (2013–2017)
CF Salmantino UDS (2017–2018)
Salamanca CF UDS (2018–)

Season to season

3 seasons in Segunda División B
1 season in Segunda División RFEF
3 seasons in Tercera División

Stadium
Salamanca UDS plays in the Estadio Helmántico, with capacity for 17,341 spectators. After the dissolution of the Union Deportiva Salamanca, in the process of liquidation of the bankruptcy, the stadium went to auction, and finally was acquired by Desarrollos Empresariales Deportivos S.L., directed by the Mexican entrepreneur Miguel Alejandro Miranda which shares with Salmantino the sports project and allows them the use of the stadium.

Fans
During the summer of 2016, the club launched a campaign of subscribers directed especially to former members of the UD Salamanca, which promises to reserve the seat that they occupied during the matches of UD Salamanca once the Helmático is fully reopened (probably for the 2017–18 season). At the end of 2016 the club counted with 4,000 subscribers.

Current squad

Former coaches

Rivalries
Salamanca UDS has a strong rivalry with Unionistas de Salamanca CF, another team in the city formed after the collapse of the original club.

While Salamanca UDS claims to be the continuation of the former UD Salamanca, Unionistas was born as a homage of the former club, dissolved in 2013.

See also
Salamanca CF UDS B, reserve team

References

External links
Official website
La Preferente team profile

Association football clubs established in 2013
2013 establishments in Castile and León
Salamanca CF UDS
Sport in Salamanca
Football clubs in Castile and León

redpajama_set_name: RedPajamaWikipedia


text:
William Kupinse
University of Puget Sound
Novel Imperialisms:
Indeterminacy and Empire in The Moonstone
In The New Machiavelli, H. G. Wells's 1911 novel of social and political realism, protagonist Richard Remington meditates upon England's management of India, that jewel in the Empire's crown. What strikes him most about the colonial relationship is the effacement of any real understanding of this colonial engine of imperial wealth. "Most English people cannot even go to this land they possess; the authorities would prevent it," observes Remington. "If Messrs. Perowne or Cook organised a cheap tour of Manchester operatives, it would be stopped. No one dare bring the average English voter face to face with the reality of India, or let the Indian native have a glimpse of the English voter" (322). How then did the English citizenry encounter the greatest—in terms of geographic size, population, and economic output—of all British colonial possessions? Some of this experience would have come through second-hand accounts of those with direct experience. According to Clive Dewey, for example, the Indian Civil Service stationed about 1,000 employees in India at a given time,1 most of whom made regular trips back to England, Scotland, and Wales to attend to their families; in later years, however, the practice of settling families in Indian became more widespread (hence contributing to one definition of that vexed term Anglo-Indian), presumably reducing one motive for return visits to the home country. Another means for the English to experience India would have been through the many histories of the country that were published during the nineteenth century, such as James Mill's History of British India, first printed in three volumes in 1817, though later expanded to six volumes and reprinted widely throughout the nineteenth-century; Mill's History is interesting in that Mill—the father of John Stuart Mill—wrote his voluminous work without having ever visited India. Newspapers of course offered yet another means for the English to encounter India, though depending on the date and one's choice of texts, fiction might prove more factually reliable than, say, the Times' infamous reporting of the 1857 Indian Mutiny, in which fabricated reports of the systematic torture and rape of English women were presented as truth.2
As I've just suggested, novels offered a potent fourth means for English citizens to encounter an India that most of them could never visit personally, and it is not clear that as a fictive genre the novel is at any inherent disadvantage when compared with the secondhand anecdotes of returned civil servants, a history written by an author who had never visited the country of which wrote, and demonstrably false newspaper reportage intended to further the goals of the East India Company and Imperial Britain. This is not to claim that all novels representing India possess greater veracity than the other sources I've cited. To continue with the subjects of the Indian Mutiny, many of the flurry of novels that responded to the uprising simply reproduced the tales of native savagery that characterized the Times' sensational reportage, as did James Grant's First Love and Last Love: A Tale of the Indian Mutiny (1868). Even historical fiction more sympathetic to the plight of Indians, such as Flora Annie Steel's On The Face of the Waters (1896), had difficulty transcending the dominant cultural narrative of native savagery; as Jenny Sharpe notes, Steel's "novel does not break with a colonial logic that explains the British retribution as a response to the massacre of innocents" (101).
It is at this point that I wish to turn to this essay's central text: Wilkie Collins's The Moonstone,3 published, like Grant's First Love and Last Love, in 1868. Although Collins's novel does not concern itself with the history of the Mutiny, most critics regard The Moonstone as a response to that colonial history, albeit one in which the details of the Mutiny are sublimated into a tale of colonial larceny set in an earlier period. Constructed out of a series of multiple personal narratives, The Moonstone begins with the storming of Seringapatam in 1798, during which military officer John Herncastle steals the remarkable yellow diamond of the novel's title from the forehead of an unidentified deity in a "Hindoo" shrine, committing a triple murder of Indian guards in the process. The novel then jumps to 1848, whereupon the narration is picked up largely by Gabriel Betteredge, the elderly head servant of the Verinder estate. Betteredge recounts the return of Franklin Blake, cousin to the Verinders, who has not visited the estate since his childhood; curiously, Blake is preceded by three itinerant Indian jugglers and a young English boy, whom Betteredge dismisses. Blake arrives bearing the yellow diamond stolen a half-century earlier by Herncastle, who has bequeathed it in his will to the young Rachel Verinder, to be presented to her as a gift on her next birthday following Herrncastle's death. On the evening following Rachel's birthday dinner, the Moonstone is stolen from the Indian cabinet in Rachel's bedroom, in which she has placed it for safekeeping. The remainder of the novel traces the characters on whom suspicion is variously placed: the reformed thief turned housemaid Rosanna Spearman; another Verinder cousin, the pious charity organizer Godfrey Ablewhite; Rachel herself; and Franklin Blake.
A recap of the main events of the Indian Mutiny may be helpful in assessing the claims of Collins scholars that The Moonstone is responding to an event that occurred a decade before the novel's publication. Alternately called the Sepoy Mutiny for the Indian troops who rebelled against their British commanders, the Indian Mutiny began with troops' reaction to rumors that the new cartridges—whose ends needed to be bitten off before inserting them in their Enfield rifles—were lubricated with a mixture of cow and pig fat, thus defiling both Hindus and Muslims. Rebellions occurred in Dum-Dum, Barrackpore, Ambala, and Meerut, from where the sepoys marched to Delhi, killing British and Europeans and installing their own leader. The slaughter of English woman and children at Cawnpore after the betrayal of British soldiers occasioned the greatest outrage on behalf of the English; that the bodies of the murdered women and children were subsequently thrown down a well did much to encourage the hyperbolic stories of rape and torture reported in the Times and retold in Grant's First Love and Last Love. While Cawnpore would serve as the symbolic center of Mutiny atrocities, the rebellion itself continued for more than six months, with major uprisings occurring in Lucknow and other cities. British suppression of the rebellion spared no use of force, and British soldiers committed atrocities exceeding even those the Indians at Cawnpore were purported to have committed—reports that were later established to be false.4
In connecting The Moonstone to the Mutiny, a relatively recent trend in Collins scholarship,5 critics have focused on Collins's sympathetic stance toward India, particularly when compared with that of his friend and editor (of Household Words, to which he often contributed), Charles Dickens. Additionally, at the time of Collins's writing of The Moonstone, much media attention was focused on commemorating the tenth anniversary of the Mutiny.6 Generally, the critical narrative that has been constructed suggests that in focusing on the storming of Seringapatam as an anachronistic displacement of the Mutiny, Collins was able to critique the British colonial practices that had motivated the sepoys to rebel. Thus, according to this reading, Collins sides with those present-day historians who see the Mutiny not as a disproportionate reaction to the greasing of munitions cartridges, but as India's first war of emancipation from the British.
In following this line of inquiry in which Collins's novel is understood as a critique of British imperialism, critics have arrived at important insights. In his essay "Outlandish English Subjects in The Moonstone," Timothy L. Carens explores how the novel reveals as uncanny the "savagery" inherent in English familial structure and religious practice.7 Similarly, in "'Dirty Linen': Legacies of Empire in Wilkie Collins's The Moonstone," Melissa Free sees Collins's project as dismantling the false binaries of imperial center and colony; according to Free, "it is the thick overlay of empire and home that we a truly examining, (a portrayal) in which home is not merely affected by but
effecting empire" (342).
In both Carens' and Free's essays, as in most scholarship focusing on the colonial dimensions of The Moonstone, Collins's character Ezra Jennings serves a central role. Introduced late in the novel, Jennings is the assistant to Dr. Candy, who in turn is an attendee of Rachel Verinder's birthday dinner the evening the diamond is stolen; as Candy has since lapsed into an illness depriving him of his mental acumen, Jennings's role becomes all the more important. When Franklin Blake first encounters him, he perceives Jennings as a striking figure. Appearing much older than his years, Jennings bears a "complexion [ ] of a gypsy darkness," and his "nose present[s] the fine shape and modeling so often found among the ancient people's of the East, so seldom visible among the newer races of the West." Most noteworthy is Jennings's hair,
which by some freak of Nature, had lost its color in the most startling and capricious manner. Over the top of his head it was still the deep black which was its natural color. Round the sides of his head—without the slightest gradation of gray to break the force of the extraordinary contrast—it had turned completely white. The line between the two colors preserved no sort of regularity. At one place, the white hair ran up into the black; at another, the black hair ran down into the white. (343-44)
Predictably, critics have seen Jennings's unusual "piebald" hair as symbolizing a dichotomy of Englishness/Otherness, as Tamar Heller does in her afterword to the collection of Collins essays Reality's Dark Light (in which she observes—shockingly—that two BBC adaptations have chosen to omit this detail from their productions).8 If, as a symbol of hybridity, Jennings depiction begins with his physiognomy, skin, and hair, it hardly ends there, and the complexity of his symbolic otherness is overdetermined. Indeed, although Jennings himself makes clear that he is of mixed ethnicity, it is unclear exactly what exactly his heritage is. "I was born and partly brought up in one of our colonies," he tells Franklin Blake. "My father was an Englishman, but my mother—we are straying far from our subject," he finishes inconclusively. The novel does give some hints at his possible Indianness; in addition to the novel's illustrations, which render Jennings and the Indian jugglers similarly, Jennings's addiction to opium establishes a symbolic connection to India. As Lillian Nader has observed, the British relationship to Indian-grown opium was a scandal that undermined the moral claims of imperialism; India was the source of British opium illicitly imported to China until the British forced China to make such importation legal in the Treaty of Tiensen.9 (Interestingly enough, critics seem to overlook the role that tobacco plays is the novel, however; as I will discuss shortly, cessation of its use becomes a major plot device that connects Blake to America—tobacco being a main export of England's former, North American colonies—just as opium connects Jennings to India. Thus the twin substances of tobacco and opium serve as synecdoches for colonies past and present.)
Jennings's indeterminate cultural hybridity would be a mere footnote in the novel, were it not for the central role he plays in untangling The Moonstone's plot. Indeed, Jennings is both doctor and detective, as Ronald R. Thomas has demonstrated, in an era in which "criminal investigation [moved] from the teeming streets of the crime scene to the chemical interactions that take place in the scientific laboratory."10 In one of the novel's surprising revelations, Franklin Blake learns that a nightshirt marked with paint that attests to the identity of the diamond's thief —which the lovelorn Roseanna Spearman had concealed—is none other than his own; later, Rachel Verinder tells him that with her own eyes she saw him steal the diamond on the night of her birthday. Apparently stunned by these revelations, Blake re-encounters Jennings, who makes a surprising revelation of his own. While nursing the ailing Dr. Candy back to health, Jennings took down in shorthand the doctor's fevered ramblings—which the novel includes as a verbatim transcript—and reconstructed them into a coherent narrative. I reproduce the whole of Candy's fragmentary speech and Jennings's reconstruction of it for reasons that will become apparent shortly.
"….Mr. Franklin Blake….and agreeable….down a peg….medicine….confesses….sleep at night….tell him….out of order….medicine….but he tells me….and groping in the dark means one and the same thing….all the company at the dinner-table….I say….groping after sleep….nothing but medicine….he says….leading the blind….know what it means….witty….a night's rest in spite of his teeth…Lady Verinder's medicine-chest….five-and-twenty minims….without his knowing it….out, Mr. Candy….excellent….without it….down on him….truth….something besides….excellent….dose of laudanum, sir….bed….what….medicine now." (408)
'….Mr. Franklin Blake is clever and agreeable, but he wants taking down a peg when he talks of medicine. He confesses that he has been suffering from want of sleep at night. I tell him that his nerves are out of order, and that he ought to take medicine. He tells me that taking medicine and groping in the dark mean one and the same thing. This before all the company at the dinner-table. I say to him, you are groping after sleep, and nothing but medicine can help you find it. He says to me, I have heard of the blind leading the blind, and now I know what it means. Witty—but I can give him a night's rest in spite of his teeth. He really wants sleep; and Lady Verinder's medicine chest is at my disposal. Give him five-and-twenty minims of laudanum to-night, with out his knowing it; and then call to-morrow morning. 'Well, Mr. Blake, will you try a little medicine today? You will never sleep without it.'—'There you are out, Mr. Candy: I have had an excellent night's rest without it.' Then come down on him with the truth! 'You had a dose of laudanum, sir, before you went to bed. What do you say to the art of medicine now?' (409)
According to Jennings's reconstruction of Candy's incoherent speech into comprehensible narrative, a recalcitrant Franklin Blake—suffering from insomnia as a result of quitting cigar smoking because Rachel Verinder disliked the habit—was drugged with laudanum on the recommendation of Dr. Candy (administered, we later learn, by Godfrey Ablewhite). Jennings further theorizes that Blake, in a somnambulant trance brought about by opium consumption and nicotine deprivation, himself removed the diamond, probably intending to move it from the Indian cabinet in which is was stored to a more secure location.
To prove the validity of his textual reconstruction, Jennings proposes an experiment: that the crime-scene of the diamond's theft be reconstructed as accurately as possible, with the household furnishings arranged as before, the lapsed smoker Blake once again quitting cigars cold-turkey, an appropriate dose of opium administered, and a substitute gem placed in the Indian cabinet. As scientific justification for his forensic experiment, and to convince Blake to cooperate in the unusual undertaking, Jennings appeals to medical authority to support the theory that the brain records all sensory information, which can be recalled under proper circumstances. After long preparations, the "experiment" is launched, and it proves a partial success. Primed with a discussion of the diamond and forty minims of laudanum (the increased amount designed to compensate for his taking the drug knowingly), Blake falls asleep, but soon arises mumbling that "[t]he Indians may be hidden in the house" (446), and retrieves the mock diamond from the cabinet, only to let it fall to the ground.
Jennings considers the experiment a qualifies success, in that it suggests that Blake did in fact take the diamond in hand the night of its theft, but it takes the rest of the novel for the plot to spin out fully. When it does, Godfrey Ablewhite is found dead in a locked lodging room of a tavern, disguised as a sailor in "swarthy" (468) makeup, a wig, and false beard. Evidently the victim of murder by the three Indian jugglers—a second generation of Hindu priests charged with the Moonstone's recovery—Ablewhite had retrieved the diamond from the money lender with whom he had secured it against a loan designed to disguise his embezzlement from a trust he managed. Sergeant Cuff, a renowned Scotland Yard detective who earlier failed to crack the case of the Moonstone, comes out of retirement to provide his testimony as to the veracity of Jennings's surmises. Interviewing the money-lender, Mr. Luker, Cuff learns that Ablewhite earlier confessed to acquiring the Moonstone in a manner consistent with Jennings's experiment. On the night following the birthday dinner, Ablewhite awakened to see Blake taking the diamond from the cabinet—and saw through the doorway that Rachel Verinder too witnessed Blake's taking of the diamond, though she could not see Ablewhite. Mumbling, as he did in during the "experiment" that "[t]he Indians may be hidden in the house" (478), Blake in his opium-induced trance handed Ablewhite the diamond for safekeeping. Ablewhite, reflecting on this perfect opportunity to efface his debts, decided to keep the diamond, since Rachel would think Blake to be the thief, whether or not she decided to come forward with this information. Thus the story ends, save for a series of short chapters in which a recovered Dr. Candy recounts Jennings's tragic death, Gabriel Betteredge tells of the marriage and forthcoming child of Rachel Verinder and Franklin Blake, and a detective hired by Sergeant Cuff writes of losing the trail of the three Indians. An epilogue in which the India-expert Mr. Muthwaite tells of watching the Moonstone's ceremonial restoration at the Hindu shrine concludes the novel.
Or, one wonders, does it? What strikes me as notable is how readers of The Moonstone, both past and present, have been willing to accept as definitive the narrative proposed by Jennings and elaborated by Sergeant Cuff, when the novel everywhere questions the finality of such accounting. Moreover, if the novel is to be understood as commentary on imperial relations between England and India—a sublimated response to the Indian Mutiny—the issue of narrative determinacy becomes all the more crucial.
Jennings himself initially stresses the unreliability of his forensic reconstruction of the import behind Dr. Candy's fragmented speech. As he tells Blake, "these notes represent a medical and metaphysical theory," qualifying further that "these notes are of my making; there is nothing but my assertion to the contrary to guarantee that they are not fabrications" (409-10). Terming the experiment a partial success, as Jennings does, seems charitable; all that Blake has done under the influence of opium is to act out those portions of a script provided him by his conversations with Rachel Verinder and Jennings.
That Sergeant Cuff's testimony confirms Jennings's detective work hardly provides any greater credibility as to the narrative constructed. Cuff begins his contribution by stating, "allow me to remark, by way of preface, that Mr. Bruff and I, together, have found a means of forcing the money-lender to make a clean breast of it" (477). That a police sergeant has secured, through the process of interrogating a money-lender, exactly those facts he expected to find, hardly establishes credulity. His concluding remark—"This was the story told by your cousin (under pressure of necessity) to Mr. Luker" (479), who has previously told several other stories as to how he acquired the diamond, foregrounds the role of necessity—that is, the possibility of pressing motives, whether financial or juridical—in the act of confession. That Cuff reports Luker's secondhand telling of Ablewhite's account of overhearing Blake mumble "[t]he Indians may be hidden in the house" with exactly the same phrasing should make any reader suspicious, given the successive layers of mediation; any player of the game "Telephone" will attest that verbal transmission never works so flawlessly. Indeed, the second part of Cuff's preface may be truer than he intends: "We have carefully sifted the statement he [Luker] has addressed to us; and here it is at your service" (477). What sort of linguistic "sifting" have Cuff and Bruff—the comic rhyming of their surnames further undercutting any notion of authority—been engaged in?
If Jennings plays the role of forensic detective in reconstructing the events surrounding the diamond's theft, it is worth noting that that the process begins with an act of literary creation, one that as Jennings himself insists, is hardly infallible. The fragments of Dr. Candy's speech reproduced earlier, could easily be translated into other comprehensible narratives, such as the following.
"….Mr. Franklin Blake was pleasant and agreeable, at least in the early part of the evening, and I accepted his suggestion that we down a peg together when next he is in town. 'The best medicine,' he confesses. 'Helps me sleep at night.' I should tell him to go easy on that prescription, but don't wish to speak out of order. "Fine enough medicine,' I say. "The therapeutic use of liquor is recorded in the earliest annals of history.' 'All good and well, but,' he tells me, 'writing history and groping in the dark means one and the same thing.' He laughs loudly; I look around at all the company at the dinner-table and see them staring silently. 'I say,' I tell him, 'speaking of groping after sleep, I should do a bit of that myself.' 'Nothing but medicine,' he says, downing his sherry with alacrity. 'The blind leading the blind,' he blurts out; I don't know what it means, though he seems, in his besotted way, to find himself witty. If he keeps it up with the sherry, he'll get a night's rest in spite of his teeth, though I suspect it will be his head that ails him in the morning. 'Lady Verinder's medicine-chest holds a much better prescription,' I tell him: 'laudanum.' 'Five-and-twenty minims would put a man to sleep without his knowing it the next day—no hangover whatsoever.' 'Perhaps some day I'll try it out, Mr. Candy,' he tells me. 'Excellent,' I say. 'But for now I'll do without it,' Blake says, 'and with another glass of sherry instead' Well, no reason to look down on him for his timidity when it comes to opium, though the truth is, I wish I could get him to take something besides alcohol. 'Excellent,' he says, tilting back his glass. 'I'll take good sherry over a dose of laudanum, sir, any day.' I tell him that I must be getting to bed, what with my rounds to make the morrow. 'Good night,' I tell Mr. Blake. 'Go easy on that medicine now.'
I offer this experiment in textual reconstruction not to suggest alcoholism on Franklin Blake's part, but simply to stress the indeterminacy of the fragments Jennings must work with, which, given the lack of any definite confirmation, remain one conjecture among many possible conjectures. That Ablewhite ends up in possession of the diamond neither rules out nor in the possibility of collusion between himself and his cousin. Ablewhite may have resorted to embezzling to pay the debts incurred by keeping a mistress in a lavish lifestyle, but Blake has his motives, too; we learn early in the novel that "Franklin Blake's debts . . . are matters of family notoriety" (243). Objection to Blake's involvement on the grounds that he could have solved his financial problems by marrying Rachel Verinder can be anticipated by the same answer given in Ablewhite's case, namely that drawing such a large sum from the family estate so early in a marriage would be unseemly. Objection on the grounds that Blake's testimonies express real surprise at the revelations about the evidence of his nightshirt, Rachel's eyewitness testimony, or the removal of Ablewhite's disguise can be answered by asking, what thief wouldn't feign surprise? And moreover, why is Godfrey Ablewhite never asked to provide his own narrative in the early days following the theft, before he disappears from sight?
The answer may be that just as there is no perfect crime, there is no perfect crime novel—even in a book as masterful as The Moonstone, which T.S. Eliot saw as inaugurating the modern detective novel. But I take the novel's self-referentiality on the subject of narrative determinacy as evidence of yet another implicit critique. If The Moonstone is the first modern detective novel, it seems already to have incorporated the public enthusiasm for the genre it establishes. "[D]etective fever," Betteredge italicizes the desire for mystery and for the resolution of justice, a desire that Ronald R. Thomas notes finds antecedents in Poe's Inspector Dupin and in "Sergeant Witchem," the hero of short stories in Household Words (and a fictionalized version of the real Inspector Whicher of Scotland Yard (68-9, 65-6). But while this desire for mystery and resolution, for crime and justice, serves well in the world of fiction—indeed, it provided Collins a living—it serves poorly when translated into the realm of political ideology, particularly its imperial varients. Indeed, if we return to the Indian Mutiny, the problem with its reportage and popular perception was precisely this desire for a clear, determinate narrative; the narrative that was constructed was one of ungovernable natives rebelling violently and illogically against their more civilized masters, and further committing systematic outrages upon women and children, whom honor required the British forces to avenge. In The Moonstone, we have no such clarity: actions can only be adduced, motives inferred. In the opening moments of the novel, it is likely that Herncastle murdered the three Hindu guards, but the only evidence is circumstantial; similarly, the novel's conclusion that Ablewhite was smothered by the three Indians requires a framing narrative, which is provided by Ezrra Jennings, by Sergeant Cuff, and by the coroner's inquest. Collins's critique, I take it, is of a world in which the truth-function has been transferred from the Law—a shaky epistemology to begin with—and awarded to the nascent field of forensic science; as Jennings himself observes during the night of the "experiment," I saw the Law (as represented by Mr. Bruff's papers) lying unheeded on the floor" (446). Indeed, Ezra Jennings, with his first name suggesting the Biblical lawgiver, himself merely represents a new version of the Law, albeit one that has arrived in uncanny, colonial, and hybrid form.
In Imagined Communities,11 Benedict Anderson proposes that the novel and the newspaper served as the adhesive joining the millions of disparate individuals of the modern nation into the unity of an imaginary interrelationship. For Anderson, what enabled this unity was the possibility of simultaneity and the existence of empty, homogenous time, which allowed individual citizens to imagine themselves as something akin to newspaper subjects or novel characters, all enmeshed in subplots that combine to form a national whole. I would like to suggest that both newspapers and novels serve an analogous function for imperial citizens to imagine themselves as part of the collective whole of empire, and that Collins himself understood how newspapers could effect this ideological end in reportage such as the coverage of the Indian Mutiny. Central to this end is the clear exposition of the guilty and the deservedness of their punishment. In crafting his mystery novel that sought to respond to the Mutiny in the safe form of historical translation, Collins's stressed the indeterminacy of cause and effect, of colonial rebellion and imperial retribution. Faced with the growing popularity and acceptance of forensic science as the arbiter of truth, Collins's sought to stress the role that narrative framing—the tool of the fiction writer—would continue to play, even given the advent of hard science into the diverse practices of criminology; this was Collins's anodyne for the "detective fever" that Betteredge identifies. Murthwaite words at the epilogue's end reinforce this sense of indeterminacy: "How [the Moonstone] has found its way back to its wild native land—by what accident, or by what crime—may be in your knowledge, but it is not in mine" (491). Nor, indeed, is it in the knowledge of Ezra Jennings or Sergeant Cuff; the only people qualified to answer that question are the three dispersed Hindu priests and they—like the Mutineers of 1857—are no longer there to ask.
1. Clive Dewey, Anglo-Indian Attitudes: The Mind of the Indian Civil Service (Rio Grande, Ohio: Hambledon Press, 1993).
2. Jenny Sharpe, Allegories of Empire: The Figure of Women in the Colonial Text (Minneapolis: U. Minnesota Press, 1993), 66.
3. Wilkie Collins, The Moonstone, 1868 (New York: Harper, 1973).
4. For this very rough history, I have relied on Patrick Brantlinger's brief description of the Mutiny. See Patrick Brantlinger, Rule of Darkness: British Literature and Imperialism, 1830-1914 (Ithaca: Cornell UP, 1994), 200-01. For further discussion of the way in which tales of Mutiny atrocities were exaggerated and/or fabricated, see Sharpe, 67.
5. Melissa Free lists Lillian Nayder, Catherine Peters, Sue Lonhoff, and Jaya Mehta as critics who view the Moonstone's focus on the siege of Seringapatam as a displacement of the 1857 Mutiny. See Melissa Free, "'Dirty Linen': Legacies of Empire in Wilkie Collins's The Moonstone," Texas Studies in Literature and Language 48.4 (2006): 347.
6. Free, 345.
7. Timothy L. Carens, "Outlandish English Subjects in The Moonstone," in Reality's Dark Light: The Sensational Wilkie Collins, ed. Maria K. Bachman and Don Richard Cox (Knoxville: U of Tennessee Press, 2003), 239-65.
8. Tamar Heller, "Afterword: Masterpiece Theatre and Ezra Jennings's Hair," in Reality's Dark Light: The Sensational Wilkie Collins, ed. Maria K. Bachman and Don Richard Cox (Knoxville: U of Tennessee Press, 2003), 361-70.
9. Lillian Nayder, "Collins and Empire," in The Cambridge Companion to Wilkie Collins, ed. Jenny Bourne Taylor (Cambridge: Cambridge UP, 2006), 147-148.
10. Ronald R. Thomas, "The Moonstone, Detective Fiction, and Forensic Science," in The Cambridge Companion to Wilkie Collins, ed. Jenny Bourne Taylor (Cambridge: Cambridge UP, 2006), 77.
11. Benedict Andersen, Imagined Communities: Reflections on the Origin and Spread of Nationalism (London: Verso, rev. ed. 2006).


text:
\section{Introduction}
The tremendous success of deep learning in multiple areas over the last decade has really revived  the interplay between physics and machine learning, in particular neural networks~\cite{carleo2019}.  On one hand, (statistical) physics ideas~\cite{bahri2020}, such as renormalization group (RG)~\cite{lin2017}, energy landscape~\cite{ballard2017}, free energy~\cite{zhang2018}, glassy dynamics~\cite{baity2018}, jamming~\cite{geiger2019}, Langevin dynamics~\cite{feng2021}, and field theory~\cite{roberts2022},  shed some light on the interpretation of deep learning and statistical inference in general~\cite{zdeborova2016}. On the other hand, machine learning and deep learning tools are harnessed to solved a wide range of physics problems, such as interaction potential construction~\cite{behler2007}, phase transition detection~\cite{carrasquilla2017}, structure encoding~\cite{bapst2020}, physical concepts discovery~\cite{iten2020}, and many others~\cite{bedolla2020,cichos2020}.
At the very intersection of these two fields lies the restricted Boltzmann machine (RBM)~\cite{hinton2006}, which serves as a classical paradigm to investigate how an overarching perspective could benefit both sides.

The RBM uses hidden-visible connections to encode  (high-order) correlations between visible units~\cite{smolensky1986}. Its  precursor--the (unrestricted) Boltzmann machine was inspired by  spin glasses~\cite{sherrington1975,ackley1985} and is often used in the inverse Ising problem to infer physical parameters~\cite{cocco2011,aurell2012,nguyen2017}.  The restriction of hidden-hidden and visible-visible connections in RBMs allows for more efficient training algorithms, and therefore leads to recent applications in  Monte Carlo simulation acceleration~\cite{huang2017}, quantum wavefunction representation~\cite{carleo2017,melko2019}, and polymer configuration generation~\cite{yu2019}.
Deep neural networks formed by stacks of RBMs have been  mapped onto the variational RG due to their conceptual similarity~\cite{mehta2014}. RBMs are also shown to be equivalent to tensor network states from quantum many-body physics~\cite{chen2018}.
As simple as it seems, energy-based models like the RBM could eventually become the building blocks of autonomous machine intelligence~\cite{lecun2022}.

Besides the above mentioned efforts, the RBM has also been applied extensively in the study of the minimal model for second-order phase transition--the Ising model. For the small systems under investigation, it was found that RBMs with an enough number of hidden units can encode the Boltzmann distribution, reconstruct thermal quantities, and generate new Ising configurations fairly well~\cite{torlai2016,morningstar2018,dangelo2020}. The visible $\to$ hidden $\to$ visible  $\cdots$ generating sequence of the RBM can be mapped onto  a RG flow in physical temperature (often towards the critical point)~\cite{iso2018,funai2020,koch2020}. But the mechanism and power   of the RBM to capture physics concepts and principles have not been fully explored. First, in what way is  the Boltzmann distribution of the Ising model learned by the RBM? Second, can the RBM learn and even quantitatively predict the phase transition without extra human knowledge? An affirmative answer to the second question is particularly appealing, because simple unsupervised learning methods such as principal component analysis (PCA) using configuration information alone do not provide quantitative prediction for the transition temperature~\cite{wang2016,wetzel2017} and supervised learning with neural networks requires human labeling of the phase type or temperature  of a given configuration~\cite{tanaka2017,kashiwa2019}.

In this article, we report a detailed numerical study on   RBM learning of the Ising model with a system size much larger than those used previously.
The purpose is to thoroughly dissect the various parts of the RBM and reveal how each part contributes to the learning of the Boltzmann distribution of the input Ising configurations. Such understanding allows us to extract several useful machine-learning estimators or predictors for physical quantities, such as entropy and phase transition temperature. Conversely, the analysis of a physical model helps us to obtain important insights about the meaning of RBM parameters and functions, such as weight matrix, visible energy and pseudo-likelihood. Below, we first introduce our Ising datasets,  the RBM and its training protocols in Sec.~\ref{sec:model}. We then report and discuss the results about model parameters, hidden layers, visible energy and pseudo-likelihood in
Sec.~\ref{sec:result}. After the conclusion, more details about the Ising model and the RBM are provided in  Appendices.  Sample codes of the RBM are shared on the GitHub at \url{https://github.com/Jing-DS/isingrbm}.

\section{Models and Methods}
\label{sec:model}

\subsection{Dataset of Ising configurations generated by Monte Carlo simulations}
The Hamiltonian of the  Ising model with $N = L^d$ spins in a configuration ${\bf s}   = [s_1, s_2, \cdots, s_N]^T$ on a $d$-dimensional hypercubic lattice of linear dimension $L$ in the absence of magnetic field is
\begin{align}
{\mathcal H}( {\bf s} ) = -J \sum_{\langle i,j\rangle} s_i s_j
\end{align}
where the spin variable $s_i = \pm 1$ ($i=1,2,\cdots,N$),  the coupling parameter $J > 0$ (set to unity) favors ferromagnetic configurations (parallel spins) and the notation $\langle i,j  \rangle$ means to sum over nearest neighbors~\cite{cipra1987}. At a given temperature $T$, the configuration ${\bf s}$ drawn from the sample space of $2^N$ states follows the Boltzmann distribution
\begin{align}
p_T( {\bf s} ) = \frac{ e^{ - \frac{{\mathcal H}( {\bf s} ) }{k_BT} } }{Z_T}
\end{align}
where $Z_T = \sum\limits_{\bf s}  e^{ - \frac{{\mathcal H}( {\bf s} ) }{k_BT} } $ is the partition function. The Boltzmann constant $k_B$ is set to unity.

Using single-flip Monte Carlo simulations under periodic boundary conditions~\cite{newman:1999}, we generate Ising configurations for  two-dimensional ($2d$) systems ($d=2$) of $L = 64$ ($N = 4096$) at $n_T = 16$ temperatures $T=0.25, 0.5, 0.75, 1.0, \cdots, 4.0$ (in units of $J/k_B$) and for three-dimensional ($3d$)  systems $(d=3)$ of $L = 16$ ($N = 4096$) at $n_T = 20$ temperatures $T =2.5,2.75$,$3.0,3.25$,$3.5,3.75,4.0$, $4.25,4.3,4.4,4.5$,$4.6,4.7,4.75,5.0,5.25$,$5.5,5.75,6.0, 6.25$. After fully equilibrated, $M=50000$  configurations at each $T$ are collected into a dataset $D_T$ for that $T$. For $2d$ systems, we also use a dataset $D_{\cup T}$ consisting of $50000$ configurations per temperature from {\em all} $T$'s.

Analytical results about thermal quantities of the $2d$ Ising model, such as internal energy $\langle E \rangle$, (physical) entropy $S$, heat capacity $C_V$ and magnetization $\langle m\rangle$, are well known~\cite{kramers:1941,onsager1944,yang:1952,plischoke:1994}. Numerical simulation methods and results about the $3d$ Ising model have also been reported~\cite{landau2021}. Thermodynamic definitions and relations used in this work are summarized in Appendix~\ref{ap:ising}.

\subsection{Restricted Boltzmann Machine (RBM)}
The restricted Boltzmann machine (RBM) is a two-layer energy-based model with $n_h$ hidden units (or neurons)  $h_i=\pm 1$ ($i=1,2,\cdots, n_h$) in the hidden layer, whose state vector is ${\bf h} = [h_1, h_2, \cdots, h_{n_h}]^T$,  and $n_v$ visible units  $v_j=\pm 1$ ($j=1,2,\cdots,n_v$) in the visible layer, whose state vector is ${\bf v} = [v_1, v_2, \cdots, v_{n_v}]^T$  (Fig.~\ref{fig:rmb})~\cite{fischer2012}. In this work, the visible layer is just the Ising configuration vector, i.e. ${\bf v} =  {\bf s}$, with  $n_v = N$. We choose binary unit $\{-1, +1 \}$ (instead of $\{0 ,1 \}$) to better align with the  definition of Ising spin variable $s_i$.

The  total energy  $E_{\boldsymbol{\theta}}({\bf v}, {\bf h})$ of the RBM  is defined as
\begin{equation}
\begin{aligned}
E_{\boldsymbol{\theta}}({\bf v}, {\bf h}) &= - {\bf b}^T {\bf v} - {\bf c}^T {\bf h} - {\bf h}^T{\bf W} {\bf v} \\
&=   -   \sum\limits_{j=1}^{n_v} b_j  v_j - \sum\limits_{i=1}^{n_h} c_i h_i  -\sum\limits_{i=1}^{n_h}  \sum\limits_{j=1}^{n_v} W_{ij} h_i v_j
\end{aligned}
\end{equation}
where ${\bf b} = [b_1, b_2, \cdots, b_{n_v}]^T$ is the visible bias, ${\bf c} = [c_1, c_2, \cdots, c_{n_h}]^T$ is the hidden bias and
\begin{equation}
{\bf W}_{n_h \times n_v} =
\begin{bmatrix}
-{\bf w}_1^T-\\
-{\bf w}_2^T-\\
\vdots\\
-{\bf w}_{n_h}^T-\\
\end{bmatrix}
=
\begin{bmatrix}
| & | & & |\\
{\bf w}_{:, 1} & {\bf w}_{:, 2}  & \cdots & {\bf w}_{:, n_v} \\
| & | & & |
\end{bmatrix}
\end{equation}
 is the interaction weight matrix between visible and hidden units. Under this notation, each row vector ${\bf w}_{i}^T$ (of dimension $n_v$) is a {\em filter} mapping from the visible state ${\bf v}$ to a hidden unit $i$ and each column vector  ${\bf w}_{:, j}$ (of dimension $n_h$) is an {\em inverse filter} mapping from  the hidden state ${\bf h}$ to a visible unit $j$.
All parameters are collectively written as ${\boldsymbol\theta} = \{ {\bf W}, {\bf b}, {\bf c} \}$.``Restricted'' refers to the lack of interaction between   hidden units or between   visible units.
 \begin{figure}
\includegraphics[width=0.45\textwidth]{rbm.png}
\caption{A restricted Boltzmann machine (RBM) with $n_h=6$ hidden units and $n_v = 9$ visible units. Model parameters ${\boldsymbol\theta} = \{ {\bf W}, {\bf b}, {\bf c} \}$ are represented by connections. A filter ${\bf w}_{1}^T$  from visible units to the first hidden unit is highlighted by red (light color) connections.}
\label{fig:rmb}
\end{figure}

The joint distribution for an overall state $({\bf v}, {\bf h})$ is
\begin{equation}
p_{\boldsymbol \theta}({\bf v}, {\bf h}) = \frac{e^{- E_{\boldsymbol \theta}({\bf v}, {\bf h})}}{Z_{\boldsymbol \theta}}
\end{equation}
where the partition function of the RBM
\begin{equation}
Z_{\boldsymbol \theta} = \sum_{\bf v} \sum_{\bf h} e^{- E_{\boldsymbol \theta}({\bf v}, {\bf h})}.
\end{equation}
The learned {\em model distribution} for visible state ${\bf v}$ is from   marginalization of  $p_{\boldsymbol \theta}({\bf v}, {\bf h})$,
\begin{equation}
p_{\boldsymbol \theta}({\bf v})  = \sum_{\bf h} p_{\boldsymbol \theta}({\bf v}, {\bf h})  =  \frac{1}{Z_{\boldsymbol \theta}}  e^{- {\mathcal E}_{\boldsymbol \theta}({\bf v})},
\end{equation}
where the {\em visible energy}--an effective energy for visible state ${\bf v}$ (often termed as ``free energy'' in machine learning literature),
 \begin{equation}
\begin{aligned}
 {\mathcal E}_{\boldsymbol \theta}({\bf v})
&= -{\bf b}^T {\bf v} - \sum\limits_{i=1}^{n_h}  \ln \left( e^{   - {\bf w}_i^T {\bf v} - c_i  }  +  e^{    {\bf w}_i^T {\bf v} + c_i     } \right)
\end{aligned}
\end{equation}
 is  defined according to $e^{- {\mathcal E}_{\boldsymbol \theta}({\bf v}) } = \sum\limits_{\bf h} e^{- E_{\boldsymbol \theta}({\bf v}, {\bf h})}$ such that
$Z_{\boldsymbol \theta} = \sum\limits_{\bf v}  e^{- {\mathcal E}_{\boldsymbol \theta}({\bf v})}.$
See Appendix~\ref{ap:rbm} for a detailed derivation.

The conditional distributions to generate ${\bf h}$ from ${\bf v}$,  $p_{\boldsymbol \theta}({\bf h}|{\bf v})$, and to generate ${\bf v}$ from ${\bf h}$,  $p_{\boldsymbol \theta}({\bf v}|{\bf h})$, satisfying $  p_{\boldsymbol \theta}({\bf v},{\bf h}) = p_{\boldsymbol \theta}({\bf h}|{\bf v})  p_{\boldsymbol \theta}({\bf v}) =  p_{\boldsymbol \theta}({\bf v}|{\bf h})p_{\boldsymbol \theta}({\bf h})$, can be written as products
\begin{equation}
\begin{aligned}
p_{\boldsymbol \theta}({\bf h}|{\bf v})  &= \prod\limits_{i=1}^{n_h} p_{\boldsymbol \theta}(h_i | {\bf v}) \\
 p_{\boldsymbol \theta}({\bf v}|{\bf h}) &= \prod\limits_{j=1}^{n_v} p_{\boldsymbol \theta}(v_j | {\bf h})
 \end{aligned}
\end{equation}
because $h_i$ are independent from each other (at fixed ${\bf v}$) and $v_j$ are independent from each other (at fixed ${\bf h}$).
It can be shown that
\begin{equation}
\label{eq:cond_prob}
\begin{aligned}
p_{\boldsymbol \theta}(h_i = 1 | {\bf v}) &= \sigma\left( 2(c_i +  {\bf w}_{i}^T {\bf v} ) \right) \\
p_{\boldsymbol \theta}(h_i = -1 | {\bf v}) & =  1- \sigma\left( 2( c_i +  {\bf w}_{i}^T {\bf v} ) \right) \\
p_{\boldsymbol \theta}(v_j = 1 | {\bf h}) &= \sigma\left( 2 ( b_j +  {\bf h}^T {\bf w}_{:,j} ) \right) \\
 p_{\boldsymbol \theta}(v_j = -1 | {\bf h}) &= 1- \sigma\left( 2( b_j +  {\bf h}^T {\bf w}_{:,j} ) \right)
\end{aligned}
\end{equation}
where the sigmoid function $\sigma(z) = \frac{1}{1+e^{-z}}$ (Appendix~\ref{ap:rbm}).

\subsection{Loss function and training of RBMs}
Given the dataset $D = [ {\bf v}_1, {\bf v}_2, \cdots, {\bf v}_M ]^T$ of $M$ samples generated independently from the identical {\em data distribution} $p_D({\bf v})$ (${\bf v} \stackrel{\text{i.i.d.}}{\sim} p_D({\bf v})$), the  goal of RBM learning is to find a model distribution $p_{\boldsymbol \theta}({\bf v})$ that approximates $p_D({\bf v})$. In the context of this work, the data samples ${\bf v}$'s are Ising configurations and the data distribution $p_D({\bf v})$ is or is related to the Ising Boltzmann distribution $p_T( {\bf s} )$.

Based on maximum likelihood estimation, the optimal parameters ${\boldsymbol \theta}^* =  \arg \min\limits_{\boldsymbol \theta}  {\mathcal L}({\boldsymbol \theta}) $ can be found by minimize the negative log likelihood
\begin{equation}
\label{eq:L}
 {\mathcal L}({\boldsymbol \theta}) = \langle -\ln p_{\boldsymbol \theta}({\bf v}) \rangle_{{\bf v}\sim p_D} = \langle  {\mathcal E}_{\boldsymbol \theta}({\bf v})  \rangle_{{\bf v}\sim p_D}  + \ln Z_{\boldsymbol \theta}
\end{equation}
which serves as the {\em loss function} of RBM learning. Note that the partition function $Z_{\boldsymbol \theta}$ only depends on the model but not on data. Since the calculation of  $Z_{\boldsymbol \theta}$ involves summation over all possible   $({\bf v}, {\bf h})$ states, which is not feasible, $ {\mathcal L}({\boldsymbol \theta})$ can not be evaluated exactly, except for very small systems~\cite{oh2020}.  Approximations have to be made, for example, by  mean-field calculations~\cite{huang2015}.  An interesting feature of the RBM is that, although the actual loss function $  {\mathcal L}({\boldsymbol \theta})$ is not accessible, its gradient
\begin{equation}
 \nabla_{\boldsymbol \theta}   {\mathcal L}({\boldsymbol \theta}) = \langle  \nabla_{\boldsymbol \theta}  {\mathcal E}_{\boldsymbol \theta}({\bf v})  \rangle_{{\bf v}\sim p_D}  - \langle  \nabla_{\boldsymbol \theta}  {\mathcal E}_{\boldsymbol \theta}({\bf v})  \rangle_{{\bf v}\sim p_\theta}
\end{equation}
can be sampled, which enables a gradient descent learning algorithm.  From step $t$ to step $t+1$,  model parameters are updated with learning rate $\eta$ as
\begin{equation}
{\boldsymbol \theta}_{t+1} =  {\boldsymbol \theta}_t - \eta  \nabla_{\boldsymbol \theta}   {\mathcal L}({\boldsymbol \theta}_t).
\end{equation}

To evaluate the loss function, we use its approximate -- the pseudo-(negative log)likelihood~\cite{besag1975}
\begin{equation}
\widetilde{\mathcal L}({\boldsymbol \theta}) = \left \langle-  \sum_{i=1}^{n_v} \ln p_{\boldsymbol \theta} (v_i | v_{j\ne i}) \right \rangle_{{\bf v}\sim p_D}
\approx  {\mathcal L}({\boldsymbol \theta})
\end{equation}
where the notation
\begin{equation}
\begin{aligned}
p_{\boldsymbol \theta} (v_i | v_{j \ne i}) &= p_{\boldsymbol \theta} (v_i | v_j ~ {\rm for} ~j \ne i) \\
& =  \frac{ e^{- {\mathcal E}_{\boldsymbol \theta}({\bf v})}   }{  e^{- {\mathcal E}_{\boldsymbol \theta}({\bf v})}  + e^{- {\mathcal E}_{\boldsymbol \theta}( [v_1,  \cdots, -v_i, \cdots, v_{n_v}])} }
\end{aligned}
\end{equation}
is the conditional probability for component $v_i$ given that all the other components  $v_j$ $(j \ne i)$ are fixed.
Practically, to avoid the time-consuming sum over all visible units $\sum\limits_{i=1}^{n_v} $, it is suggested to randomly sample one $i_0 \in \{ 1, 2, \cdots, n_v \}$ and estimate that
\begin{equation}
\widetilde{\mathcal L}({\boldsymbol \theta}) \approx \left \langle-   n_v \ln p_{\boldsymbol \theta} (v_{i_0} | v_{j \ne i_0}) \right \rangle_{{\bf v}\sim p_D},
\end{equation}
if all the visible units are on average translation-invariant~\cite{lisa2018}.
To monitor the reconstruction error, we also calculate the cross entropy ${\rm CE}$ between the initial configuration ${\bf v}$ and the conditional probability $p_{\boldsymbol \theta}( {\bf v}'  | {\bf h})$ for reconstruction ${\bf v} \stackrel{p_{\boldsymbol \theta}( {\bf h}  | {\bf v})}{\longrightarrow} {\bf h} \stackrel{p_{\boldsymbol \theta}( {\bf v}'  | {\bf h})}{\longrightarrow} {\bf v}'$ (See Appendix~\ref{ap:rbm_train} for definition).

For both $2d$ and $3d$ Ising systems, we first train single temperature RBMs ($T$-RBM). $M=50000$ Ising configurations at each $T$  forming a dataset $D_T$ are used to train one model such that there are $n_T$ $T$-RBMs in total. While $n_v = N$, we try various number of hidden units with $n_h = 400, 900, 1600, 2500$ in $2d$ and $n_h = 400, 900, 1600$ in $3d$. For $2d$ systems, we also train an all temperature RBM ($\cup T$-RBM) for which $50000$ Ising configurations per temperature are drawn to compose a dataset $D_{\cup T}$ of $M = 50000 n_T = 8\times 10^5$ samples.  The number of hidden units  for this $\cup T$-RBM is $n_h = 400, 900, 1600.$  Weight matrix ${\bf W}$ are initialized with Glorot normal
initialization~\cite{glorot2010} (${\bf b}$ and ${\bf c}$ are initialized as zero). Parameters are optimized with the stochastic gradient descent algorithm of  learning rate $\eta = 1.0\times 10^{-4}$ and batch size 128. The negative phase (model term) of the gradient $\langle  \nabla_{\boldsymbol \theta}  {\mathcal E}_{\boldsymbol \theta}({\bf v})  \rangle_{{\bf v}\sim p_\theta}$ is calculated using  CD-k Gibbs sampling  with $k=5$. We stop the training  until $\widetilde{\mathcal L}$ and CE converge, typically at 100-2000   epochs (see Supplemental Material).
Three Nvidia GPU cards (GeForce RTX 3090 and 2070) are used to train the model, which takes about two mins per epoch for a $M=50000$ dataset.

\section{Results and Discussion}
\label{sec:result}
In this section, we investigate how the RBM uses its weight matrix ${\bf W}$ and hidden layer ${\bf h}$ to encode the Boltzmann distributed states of the Ising model, and what physical information can be extracted from machine learning concepts such as visible energy and loss function.

\subsection{Filters and inverse filters}
\begin{figure
\includegraphics[width=0.45\textwidth]{combined_parameters.jpg}
\caption{  Probability density function (PDF) of the distribution of (a) $W_{ij}$,  (b) $b_j$, and (c)  $c_i$ of $T$-RBMs with $n_h=400$ hidden units at temperatures below, close to and above $T_c$.}
\label{fig:ising2d_W_b_c}
\end{figure}

It can be verified that the trained weight matrix elements $W_{ij}$ of a $T$-RBM follows a Gaussian distribution of zero mean with largest variance at $T \sim T_c$ (Fig.~\ref{fig:ising2d_W_b_c}a)  The high temperature distribution here is different from the uniform distribution observed in Ref.~\cite{torlai2016}. According to Eq.~(\ref{eq:cond_prob}), the biases $c_i$ and $b_j$ can be associated with the activation threshold of a hidden unit and a visible unit, respectively. For example,  whether a hidden unit is activated ($h_i = +1$) or anti-activated  ($h_i = -1$) depends on whether the incoming signal ${\bf w}^T_i {\bf v}$ from all visible units exceeds the threshold $-c_i$. The values of $c_i$ (and $b_j$) are all close to zero and  are  often negligible in comparison  with the total incoming signal ${\bf w}^T_i {\bf v}$ (and ${\bf h}^T {\bf w}_{:,j}$) (see Supplemental Material for the results of constrained RBMs where all biases are  set to zero). The distribution of  $c_i$ and $b_j$ should in principle be symmetric about zero (Fig.~\ref{fig:ising2d_W_b_c}b-c).  A non-zero mean can be caused by an unbalanced dataset with unequal number of $m>0$ and $m<0$ Ising configurations.  The corresponding filter or inverse filter sum may also be distributed with a non-zero mean in order to compensate the asymmetric bias as will be shown next.

\begin{figure
\includegraphics[width=0.45\textwidth]{ising2d_W400_sample.jpg}
\includegraphics[width=0.45\textwidth]{combined_2d_Wfilter.jpg}
\includegraphics[width=0.45\textwidth]{combined_2d_WTfilter.jpg}
\caption{$T$-RBMs  with  $n_h=400$  for the $2d$ Ising model at temperature $T=1.0$, $2.25$, and $3.5$. (a) Five sample filters ${\bf w}^T_i$  at each temperature. The colorbar range is set to be within about two standard deviations of the distribution. (b) PDF of the distribution of the  $n_h = 400$ filter sums (normalized by $n_v$). Inset: variance $ \left \langle \left| \sum\limits_{j=1}^{n_v}  W_{ij}  \right|^2 \right\rangle - \left \langle \left| \sum\limits_{j=1}^{n_v}  W_{ij}   \right| \right\rangle^2$ of the filter sum as a funciton of temperature. (c) PDF of the distribution of the  $n_v = 4096$ inverse filter sums (normalized by $n_h$). Inset: correlation between a pair of inverse filters ${\bf w}_{:,j}$ and ${\bf w}_{:,j'}$ (normalized by auto-correlation) as a function of spin-spin distance $r_{jj'}$. }
\label{fig:ising2d_W_sample}
\end{figure}

Since ${\bf v} = {\bf s}$ is an Ising configuration with $\pm 1$ units in our problem, ${\bf w}^T_i {\bf v}$  will be more positive (or negative) if  the components of ${\bf w}^T_i$ better match (or anti-match) the signs of spin variables. In this sense, we can think of  ${\bf w}^T_i$ as a filter extracting certain patterns in Ising configurations. Knowing the representative spin configurations of the Ising model below, close to and above the critical temperature $T_c$, we expect that ${\bf w}^T_i$ ($i=1,2,\cdots, n_h$) wrapped into a $L^d$ arrangement exhibits similar features.   In Fig.~\ref{fig:ising2d_W_sample}a, we show sample filters of $T$-RBMs with $n_h = 400$ trained for the $2d$ Ising model at three temperatures $T=1.0, 2.25$ and $3.5$ (see Supplemental Material for more examples of filters). At low $T$, the components of ${\bf w}^T_i$ tend to be mostly positive (or negative) matching the spin up (or spin down) configurations in the ferromagnetic phase. At high $T$, filters ${\bf w}^T_i$ possess strip domains consisting of roughly  equal number of well-mixed positive and negative components, like Ising configurations during spinodal decomposition. Close to $T_c$, ${\bf w}^T_i$  patterns vary dramatically from each other,  in accord with the large critical fluctuation. In particular, some even exhibit hierarchical clusters of various sizes. The element sum of the filter -- {\em filter sum} ${\rm sum}({\bf w}_i^T ) = \sum\limits_{j=1}^{n_v}  W_{ij} $, plays the similar role as the magnetization $m$. The distribution of all the $n_h$ filter sums at each $T$ changes with increasing temperature as the Ising magnetization changes, from bimodal to unimodal with largest variance at $T_c$ (Fig.~\ref{fig:ising2d_W_sample}b). This suggests that the peak of  the variance $ \left \langle \left| \sum\limits_{j=1}^{n_v}  W_{ij} \right|^2 \right\rangle - \left \langle \left| \sum\limits_{j=1}^{n_v}  W_{ij} \right| \right\rangle^2$ as a function of temperature coincides with the Ising phase transition (inset of Fig.~\ref{fig:ising2d_W_sample}b). More detailed results about $2d$ and $3d$ Ising model are in Supplemental Material.

\begin{figure}
\includegraphics[width=0.4\textwidth]{ising2d_all_W400_sample.jpg}
\includegraphics[width=0.45\textwidth]{combined_2d_all_parameters.jpg}
\caption{The $\cup T$-RBM  with  $n_h=400$  for the $2d$ Ising model. (a) Four sample filters ${\bf w}^T_i$. (b) PDF of the distribution of $W_{ij}$,  $b_j$, and $c_i$. (c) PDF of the distribution of the  $n_h = 400$ filter sums and the $n_v = 4096$ inverse filter sums. }
\label{fig:ising2d_all_W_sample}
\end{figure}

When a  hidden layer ${\bf h}$ is provided, the RBM reconstructs the visible layer ${\bf v}$ by applying the $n_v$ inverse filters ${\bf w}_{:, j}$ ($j=1,2,\cdots,n_v$) on ${\bf h}$. The distribution of the {\em inverse filter sum} ${\rm sum}({\bf w}_{:,j}) = \sum\limits_{i=1}^{n_h}  W_{ij} $ is Gaussian with a mean close to zero (Fig.~\ref{fig:ising2d_W_sample}c), where a large deviation from zero mean is accompanied by a non-zero average bias $ \sum\limits_j b_j /n_v$ as mentioned above (Fig.~\ref{fig:ising2d_W_b_c}b). We find that this is a result of the unbalanced dataset which has $\sim 60\%$ $m<0$ Ising configurations. Because the activation probability of a visible unit $v_j$ is determined by ${\bf w}_{:, j}$, the correlation between visible units (Ising spins) is reflected in the correlation between inverse filters. This is equivalent to the analysis of the $n_v\times n_v$ matrix ${\bf W}^T {\bf W}$ as in Ref.~\cite{iso2018}, whose entries are inner product ${\bf w}_{:,j}^T {\bf w}_{:,j'}$ of inverse filters.  We can therefore locate the Ising phase transition by identifying the temperature with the strongest correlation among ${\bf w}_{:, j}$'s, e.g. the peak of ${\bf w}_{:, j}^T {\bf w}_{:, j'}$ at a given distance $r_{jj'}$ (inset of Fig.~\ref{fig:ising2d_W_sample}c). See Supplemental Material for results in $2d$ and $3d$.

In contrast, the filters of the $\cup T$-RBM trained from $2d$ Ising configurations at all temperatures have background patterns like the high temperature $T$-RBM (in the paramagnetic phase). A clear difference is that most $\cup T$-RBM filters have one large domain of positive or negative elements (Fig.~\ref{fig:ising2d_all_W_sample}a), similar as the receptive field in a deep neural network~\cite{mehta2014}. This domain randomly covers an area of the visual field of the $L\times L$ Ising configuration (see Supplemental Material for all the $n_h$ filters).  The existence of such domains in the filter causes the filter sum and the corresponding bias $c_i$ to be positive or negative with a bimodal distribution (Fig.~\ref{fig:ising2d_all_W_sample}b-c). The inverse filter sum and its corresponding bias $b_j$ still has a Gaussian distribution, although the unbalanced dataset  shifts the mean of $b_j$ away from zero.

\subsection{Hidden layer}
Whether a hidden unit uses $+1$ or $-1$ to encode a pattern of the visible layer ${\bf v}$ is randomly assigned during training. In the former case, the filter ${\bf w}^T_i$ matches the pattern (${\bf w}^T_i {\bf v}$ is positive);   in the latter case, the filter anti-matches the pattern (${\bf w}^T_i {\bf v}$ is negative).  For a  visible layer ${\bf v}$ of  magnetization $m$, the  sign of $ {\bf w}_i^T {\bf v}$ and the encoding $h_i$ is largely determined by the sign of ${\rm sum}({\bf w}_i^T )$ (Table~\ref{table:hidden}). Since the distribution of ${\rm sum}({\bf w}_i^T )$ is  symmetric about zero, the hidden layer of a $T$-RBM   roughly consists of an equal number of $+1$  and $-1$ units -- the ``magnetization'' $m_h = \frac{1}{n_h}\sum\limits_{i=1}^{n_h} h_i$ of the hidden layer is always close to zero and its average $\langle m_h \rangle \approx 0$.  The histogram of $m_h$ for all hidden encodings of   visible states is expected to be symmetric about zero (Fig.\ref{fig:ising2d_var_m_h}). We find that for the smallest $n_h$ the histogram of $m_h$ at temperatures close to $T_c$ is bimodal due to the relatively large randomness of small hidden layers. As more hidden units are added, the two peaks merge into one and the distribution of $m_h$ becomes narrower. This suggests that larger hidden layer tends to have smaller deviation from $m_h=0$.
\begin{table}[H]
\setlength\extrarowheight{4pt}
\caption{When ${\rm sum}({\bf w}_i^T )>0$, a visible layer pattern ${\bf v}$ with magnetization $m>0$ (or $m<0$) is more likely to be encoded by a hidden unit $h_i = +1$ (or $h_i = -1$). When ${\rm sum}({\bf w}_i^T )<0$, the encoding is opposite.}
\begin{center}
\begin{tabular}{ccc}
\hline
  & ${\rm sum}({\bf w}_i^T )>0$&  ${\rm sum}({\bf w}_i^T )<0$  \\
\hline
$m > 0$  & $h_i = +1$ & $h_i = -1$\\
$m < 0$  & $h_i = -1$ & $h_i = +1$\\
\hline
\end{tabular}
\end{center}
\label{table:hidden}
\end{table}

The order of the $h_i=\pm 1$ sequence in each hidden encoding ${\bf h}$ is arbitrary but relatively fixed once the $T$-RBM is trained. Permutation of hidden units together with their corresponding filters (swap rows of the matrix ${\bf W}$) results in an equivalent $T$-RBM. Examples of hidden layers of  $T$-RBMs with $n_h=400$ at different temperatures are shown in the inset of  Fig.\ref{fig:ising2d_var_m_h}, where the vector ${\bf h}$ is wrapped into a $20\times 20$ arrangement. Note that there is actually no spatial relationships between different hidden units and any apparent pattern in this $2d$ illustration is an artifact of the wrapping protocol.

\begin{figure}
\includegraphics[width=0.45\textwidth]{combined_2d_mh_T225_hist.jpg}
\caption{Histogram of $m_h$ obtained from the hidden encodings of  $M=50000$  $2d$ Ising configurations at $T=2.25$ using $T$-RBMs with various $n_h$. Inset: examples of the hidden layer of $T$-RBMs with $n_h=400$ wrapped into a $20 \times 20$ matrix at three temperatures, where $+1$/$-1$ units are represented by black/white pixels.}
\label{fig:ising2d_var_m_h}
\end{figure}

\begin{figure}
\includegraphics[width=0.4\textwidth]{combined_h_generate.jpg}
\caption{(a) Internal energy, (b) magnetization, and (c) specific heat of $2d$ Ising states reconstructed by $T$-RBMs ($n_h=400$) with the hidden layer ${\bf h}^{(0)}$ initiated according to  $p_h = ( \langle |m| \rangle +1)/2$ or $p_h = 1.0 (T\le2.0), 0.5 (T \ge 2.5), 0.75 (2.0<T<2.5)$ (stepwise). Reconstruction  by a seven-step Markov chain from random  ${\bf h}^{(0)}$ is compared (${\bf v}^{(7)}$). Analytical and Monte Carlo simulation results are also shown.  }
\label{fig:ising2d_generate}
\end{figure}

\begin{figure*
\includegraphics[width=1\textwidth]{combined_rbm_E.jpg}
\caption{Mean and variance of visible energy ${\mathcal E}_{\boldsymbol \theta}$ as a function of temperature for  $2d$ (a,c,d,f) and $3d$ (b,e) Ising models captured by $T$-RBMs (a,b,d,e) and the $\cup T$-RBM (c,f) of various hidden neurons $n_h$. Three approximate forms of visible energy for $n_h=400$ $T$-RBMs are shown in (a).}
\label{fig:combined_rbm_E}
\end{figure*}

As a generative model, a $T$-RBM can be used to produce more Boltzmann distributed Ising configurations. Starting from a random hidden state ${\bf h}^{(0)}$, this is often fulfilled by  a sequence of Markov chain moves ${\bf h}^{(0)} \to {\bf v}^{(0)} \to {\bf h}^{(1)} \to {\bf v}^{(1)} \to \cdots $ until steady state is achieved. Based on above mentioned observations, we can design an algorithm to initialize ${\bf h}^{(0)}$ that  better captures the hidden encoding of visible states (equilibrium Ising configurations), thus enables faster convergence of the Markov chain. After choosing a low temperature $T_L$ and a high temperature $T_H$, we generate the hidden layer as follows:
\begin{itemize}
    \item At low $T \le T_L < T_c$, if ${\rm sum}({\bf w}_i^T )>0$, $h_i = + 1$; if ${\rm sum}({\bf w}_i^T )<0$, $h_i = - 1$. This will be an encoding of a $m>0$ ferromagnetic configuration.  To encode of a $m<0$ ferromagnetic configuration, just flip the sign of $h_i$.
    \item At high $T \ge T_H > T_c$, randomly assign  $h_i = + 1$ or $-1$ with equal probability. This will be an encoding of a paramagnetic configuration with $m \approx 0$.
    \item At intermediate  $T_L < T < T_H$, to encode a  $m>0$ Ising configuration, if ${\rm sum}({\bf w}_i^T )>0$, assign $h_i = + 1$ with probability $p_h \in (0.5,1.0)$ and $h_i = - 1$ with probability $1-p_h$;  if ${\rm sum}({\bf w}_i^T )<0$, assign $h_i = - 1$ with probability $p_h \in (0.5,1.0)$ and $h_i = + 1$ with probability $1-p_h$. $p_h$ is a predetermined parameter and the above two algorithms  are just the special cases with $p_h = 1.0$ ($T \le T_L$) and $p_h = 0.5$ ($T \ge T_H$), respectively.  In practice, one may approximately use  $p_h = ( \langle |m| \rangle +1)/2$ or use linear interpolation within $T_L < T < T_H$, $p_h = 0.5 + 0.5(T-T_L)/(T_H - T_L)$.
\end{itemize}
Below we compare the (one-step) reconstructed thermal quantities using two different initial hidden encodings with results from a conventional multi-step Markov chain (Fig.~\ref{fig:ising2d_generate}).  The hidden encoding methods proposed here are quite reliable at low and high $T$, but less accurate at $T$ close to  $T_c$.

\begin{figure*
\centering
\includegraphics[width=1\textwidth]{combined_rbm_L.jpg}
\caption{Pseudo-likelihood $\widetilde{\mathcal L}$ per spin of $T$-RBMs (a-b) and of the $\cup T$-RBM (c) with different number $n_h$ of hidden units for the $2d$ (a,c) and $3d$ (b) Ising model in comparison with entropy $S$ and pseudo-entropy $\widetilde{S}$ per spin. Dashed lines are polynomial fittings around $T_c$.}
\label{fig:ising_L}
\end{figure*}

\subsection{Visible energy}

When a $T$-RBM for temperature  $T$ is trained,  we expect that $p_{\boldsymbol \theta}({\bf v}) \approx p_D({\bf v}) \approx   p_T({\bf s})$ -- the Boltzmann distribution at that $T$.
Although formally related to the physical energy  in the Boltzmann factor (with temperature absorbed), the visible energy $ {\mathcal E}_{\boldsymbol \theta}({\bf v}) $ of a RBM should be really considered as the  negative log (relative) probability of a visible state ${\bf v}$. For single temperature $T$-RBMs, the mean visible energy $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $ increases monotonically with temperature (except for the largest $n_h$, which might be due to overfitting) (Fig.~\ref{fig:combined_rbm_E}a-b). The value of $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $ and its trend, however, cannot be used to identify the physical phase transition. In fact,  $ {\mathcal E}_{\boldsymbol \theta}({\bf v}) $  can differ from the reduced Hamiltonian $\frac{\mathcal H({\bf s})}{k_BT}$ by an arbitrary (temperature-dependent) constant while still maintaining the Boltzmann distribution  $p_{\boldsymbol \theta}({\bf v}) \approx p_T({\bf s})$ (if the partition function $Z_{\boldsymbol \theta}$ is calibrated accordingly).

The trend of $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $ for $T$-RBMs can be understood by considering following approximate forms. First, due to the symmetry of $+1$ and $-1$, the biases $b_j$ and $c_i$ are all close to zero. A constrained $T$-RBM with zero bias has a visible energy
\begin{equation}
\label{eq:Ew}
\begin{aligned}
 {\mathcal E}_{\boldsymbol W}({\bf v})
= - \sum\limits_{i=1}^{n_h}  \ln \left( e^{   - {\bf w}_i^T {\bf v}}  +  e^{    {\bf w}_i^T {\bf v}} \right)
\end{aligned}
\end{equation}
that approximates the visible energy of the full $T$-RBM, i.e. $ {\mathcal E}_{\boldsymbol \theta}({\bf v})   \approx {\mathcal E}_{\boldsymbol W}({\bf v}) $.
Next, unless ${\bf w}_i^T {\bf v}$ is close to zero,  one of the two exponential terms in Eq.~(\ref{eq:Ew}) always dominates such that ${\mathcal E}_{\boldsymbol W}({\bf v}) \approx  \widetilde{{\mathcal E}}_{\boldsymbol W}({\bf v}) $ where
\begin{equation}
\label{eq:Ewap}
\begin{aligned}
 \widetilde{{\mathcal E}}_{\boldsymbol W}({\bf v})
= - \sum\limits_{i=1}^{n_h}  \left|{\bf w}_i^T {\bf v} \right| =  - \sum\limits_{i=1}^{n_h}  \left| \sum\limits_{j=1}^{n_v}  W_{ij} v_j \right|.
\end{aligned}
\end{equation}
Eq.~(\ref{eq:Ewap}) can further be approximated by setting ${\bf v} = {\bf 1}$ with all $v_j = +1$,  i.e. $\widetilde{{\mathcal E}}_{\boldsymbol W}({\bf v}) \approx  \widetilde{{\mathcal E}}_{\boldsymbol W}({\bf 1}) $ with
\begin{equation}
\begin{aligned}
 \widetilde{{\mathcal E}}_{\boldsymbol W}({\bf 1})
= - \sum\limits_{i=1}^{n_h}  \left| {\rm sum}({\bf w}_i^T ) \right|
= - \sum\limits_{i=1}^{n_h}  \left| \sum\limits_{j=1}^{n_v}  W_{ij} \right|.
\end{aligned}
\end{equation}
In summary, $ {\mathcal E}_{\boldsymbol W}({\bf v})$, $ \widetilde{{\mathcal E}}_{\boldsymbol W}({\bf v})$ and $\widetilde{{\mathcal E}}_{\boldsymbol W}({\bf 1})$ are all good approximations to the original ${\mathcal E}_{\boldsymbol \theta}({\bf v})  $  (Fig.~\ref{fig:combined_rbm_E}a). The increase of mean $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $ with temperature coincides with the increase of   $-\left| {\rm sum}({\bf w}_i^T ) \right|$ with temperature, which is evident from Fig.~\ref{fig:ising2d_W_sample}b. At fixed temperature, the decrease of $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $  with $n_h$ is  a consequence of the sum $\sum\limits_{i=1}^{n_h} $ in the definition of visible energy.
The variance  $\langle  {\mathcal E}_{\boldsymbol \theta}^2 \rangle - \langle  {\mathcal E}_{\boldsymbol \theta}\rangle^2$ is a useful quantity for phase transition detection, because it reflects the fluctuation of the probability $p_{\boldsymbol \theta}({\bf v})$. In both low $T$ ferromagnetic and high $T$ paramagnetic regimes, $p_{\boldsymbol \theta}({\bf v})$ is relatively homogeneous among different states. When $T$ is close to $T_c$, the variance of $p_{\boldsymbol \theta}({\bf v})$ and ${\mathcal E}_{\boldsymbol \theta}({\bf v})$ is expected to peak (Fig.~\ref{fig:combined_rbm_E}d-e). The abnormal rounded (and even shifted) peaks at large $n_h$ could be a sign of overfitting.

For the all temperature $\cup T$-RBM, the Ising phase transition can be revealed by either the sharp increase of the mean  $ \langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $  or the peak of the variance $\langle  {\mathcal E}_{\boldsymbol \theta}^2 \rangle - \langle  {\mathcal E}_{\boldsymbol \theta}\rangle^2$  (Fig.~\ref{fig:combined_rbm_E}c,f). However, this apparent detection can be a trivial consequence of the special composition of the dataset $D_{\cup T}$, which contains Ising configurations at different temperatures in equal proportion. Only configurations at a specific $T$ are fed into the model to calculate  the average quantity at that $T$. Technically, a visible state ${\bf v}$ in $D_{\cup T}$ is not subject to the Boltzmann distribution at any specific temperature. Instead, the true ensemble of $D_{\cup T}$ is a collection of $n_T$ different Boltzmann distributed subsets. Many replicas of the same or similar ferromagnetic states are in $D_{\cup T}$, giving rise to a large multiplicity, high probability and low visible energy for such states. In comparison, high temperature paramagnetic states are all different from each other, and therefore have low $p_{\boldsymbol \theta}({\bf v})$ (high $ {\mathcal E}_{\boldsymbol \theta}({\bf v})$) for each one of them. Knowing this caveat, one should be cautious when monitoring the visible energy of a $\cup T$-RBM to detect phase transition, because changing the proportion of Ising configurations at different temperatures in $D_{\cup T}$ can modify the relative probability of each state.

\subsection{Pseudo-likelihood and entropy estimation}
The likelihood $ {\mathcal L}({\boldsymbol \theta})$ defined in Eq.~(\ref{eq:L}) is conceptually equivalent to the physical entropy $S$  defined by the Gibbs entropy formula, apart from the Boltzmann constant $k_B$ difference (Appendix~\ref{ap:ising}). However, just as entropy $S$ cannot be directly sampled, the exact value of $ {\mathcal L}({\boldsymbol \theta})$ is not accessible. In order to estimate $S$, we calculate the pseudo-likelihood  $\widetilde{\mathcal L}({\boldsymbol \theta})$ instead, which is based on the mean-field like approximation $ p_{\boldsymbol \theta}({\bf v}) \approx \prod_{i=1}^{n_v} p_{\boldsymbol \theta} (v_i | v_{j\ne i})$.
A similar idea to estimate  free energy was put forward using variational autoregressive networks~\cite{wu2019}.
The true and estimated entropy of $2d$ and $3d$ Ising models using  $T$-RBMs with different  $n_h$ are shown in Fig.~\ref{fig:ising_L} (a-b).
As a comparison, we also consider a ``pseudo-entropy'' with the similar approximation
\begin{equation}
\widetilde{S} = - k_B \left\langle \sum_{i=1}^{N} p_{T} (s_i | s_{j\ne i}) \right\rangle_{{\bf s}\sim p_T} \approx S
\end{equation}
where the conditional probability
\begin{equation}
p_{T} (s_i | s_{j\ne i}) =  \frac{ e^{- \frac{{\mathcal H}({\bf s})}{k_B T}}   }{   e^{- \frac{{\mathcal H}({\bf s})}{k_B T}}   +   e^{- \frac{{\mathcal H}( [s_1,  \cdots, -s_i, \cdots, s_{N}])}{k_B T}}}
\end{equation}
and the ensemble average  $ \left\langle \cdots \right\rangle_{{\bf s}\sim p_T} $ is taken over states obtained from  Monte Carlo sampling. In both $2d$ and $3d$, $\widetilde{S}$ is lower than the true $S$, especially at high $T$, because a mean-field treatment tends to underestimate fluctuations.

While increasing model complexity by adding hidden units is usually believed  to reduce the reconstruction error, e.g. of energy and heat capacity~\cite{torlai2016,morningstar2018} (see also Supplemental Material), recent study suggests that a trade-off could exist between the accuracy of different statistical quantities~\cite{yevick2021}. Here we find that the pseudo-likelihood of $T$-RBMs with the fewest hidden units in our trials ($n_h=400$) appears to provide the best prediction for entropy. Increasing $n_h$ leads to larger deviations from the true $S$ at higher $T$.  The decreasing of  $\widetilde{\mathcal L}$ with $n_h$ at fixed temperature agrees with the trend of the visible energy. A lower ${\mathcal E}_{\boldsymbol \theta}({\bf v})$ corresponds to a higher $p_{\boldsymbol \theta}({\bf v}) $ and thus a lower $\widetilde{\mathcal L}$ according to its definition.
The surprisingly good performance of $\widetilde{\mathcal L}$  in approximating $S$ could be due to the fact that visible units $v_i$ in RBMs are only indirectly correlated through hidden units, which collectively serve as an effective mean-field on each visible unit.
We also calculate  $\widetilde{\mathcal L}({\boldsymbol \theta})$ with the all temperature  $\cup T$-RBM in $2d$ (Fig.~\ref{fig:ising_L}c).  Compared with single temperature $T$-RBMs of the same $n_h$ (Fig.~\ref{fig:ising_L}a), the $\cup T$-RBM predicts higher $\widetilde{\mathcal L}({\boldsymbol \theta})$  with considerable deviations even at low $T$. The trend of $\widetilde{\mathcal L}({\boldsymbol \theta})$ also agrees with that of $\langle {\mathcal E}_{\boldsymbol \theta}({\bf v}) \rangle $ (Fig.~\ref{fig:combined_rbm_E}c).

A knowledge about the entropy allows us to estimate the phase transition point according to the thermodynamic relation $C_V = T \frac{d S}{dT}$. We construct this estimated $C_V$ as a function of temperature using $\widetilde{\mathcal L}({\boldsymbol \theta})$ and its numerical fitting, whose peaks are expected to be located at $T_c$ (Supplemental Material). The predicted $T_c$ are compared with the results from entropy and pseudo-entropy, as well as the known exact values in Table~\ref{table:Tc}. It can be seen that single temperature $T$-RBMs capture the transition point fairly well within an error about $1$-$3\%$.
\begin{table}[H]
\setlength\extrarowheight{4pt}
\caption{$T_c$ estimated according to the peak of $T \frac{d \widetilde{\mathcal L} }{dT}$  obtained from single temperature $T$-RBMs and the all temperature $\cup T$-RBM with different number ($n_h$) of hidden units. Predictions from  numerical derivatives $T \frac{d S }{dT} $ and  $T \frac{d \widetilde{S} }{dT} $ are also shown for  comparison.}
\begin{tabular}{cccccccc}
\hline
model & $ n_h=400$&  $  900$&  $ 1600$&  $  2500$&  $ S$&  $ \widetilde{S}$ & exact \\
\hline
$2d$ $T$-RBM  & 2.240 & 2.291& 2.316& 2.367& 2.267& 2.367& 2.269\\
$2d$ $\cup T$-RBM & 2.189 & 2.163 & 2.214& - & 2.267& 2.367& 2.269 \\
$3d$ $T$-RBM & 4.444 & 4.434 & 4.444 & - & 4.390& 4.383& 4.511\\
\hline
\end{tabular}
\label{table:Tc}
\end{table}

\section{Conclusion}
In this work, we trained RBMs using  equilibrium Ising configurations in $2d$ and $3d$ collected from Monte Carlo simulations at various temperatures.  For single temperature $T$-RBMs,  the filters (row vectors) and the inverse filters (column vectors) of the weight matrix exhibit different characteristic patterns and  correlations, respectively, below, around and above the phase transition. These metrics, such as filter sum fluctuation and inverse filter correlation, can be used to locate the phase transition point. The hidden layer ${\bf h}$ on average contains an equal number of $+1$ and $-1$ units, whose variance  decreases as more hidden units are added. The sign of a particular hidden unit $h_i$ is determined by the signs of the filter sum ${\rm sum}({\bf w}_i^T )$ and the magnetization $m$ of the visible pattern. But there is no spatial pattern in the sequence of  positive and negative units in a hidden encoding.

The visible energy reflects the relative probability  of visible states in the  Boltzmann distribution. Although the mean of visible energy is not directly related to the (physical) internal energy and does not reveal a clear transition, its fluctuation which peaks at the critical point can be used to identify the phase transition. The value and trend of the visible energy can be understood from its several approximation forms, in particular, the sum of the absolute value of filter sums.  The pseudo-likelihood of RBMs is conceptually related to and can be used to estimate the physical entropy.  Numerical differentiation of pseudo-likelihood provides another estimator of the transition temperature because it provides an estimate of the heat capacity. All these predictions about the critical temperature are made by unsupervised RBM learning, for which human labeling of phase types are not needed.

 As a comparison, we also trained an all temperature $\cup T$-RBM whose dataset is a mixture of Boltzmann-distributed states over a range of temperatures. Each filter of this $\cup T$-RBM is featured by one large domain in its receptive field. Although the visible energy and pseudo-likelihood of the  $\cup T$-RBM show certain signature of the phase transition, one should be cautious that this detection could be an artifact of the composition of the dataset. Changing the proportions of Ising configurations at different temperatures could bias the probability and the transition learned by the $\cup T$-RBM.

 By extracting the underlying (Boltzmann) distribution of input data, RBMs capture the rapid (phase) transition of such distribution as the tuning parameter (temperature) is changed,  without knowledge of the physical Hamiltonian. Information about the distribution is completely embedded in the configurations and their frequencies in the dateset.  It would be interesting to see if such a general scheme of RBM learning can be extended to study other physical models of phase transition.

\begin{acknowledgments}
We thank the Duke Kunshan startup funding and the Summer Research Scholars (SRS) program for supporting this work.
\end{acknowledgments}

redpajama_set_name: RedPajamaArXiv


text:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Runtime.Serialization.Json;
using System.IO;
using System.Text;
using System.Web.Script.Serialization;

public partial class Prottype_JsonObject : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
    }

    public static string JsSerialize<T>(T obj)
    {
        JavaScriptSerializer serializer = new JavaScriptSerializer();
        string Json = serializer.Serialize(obj);
        return Json;
    }

    public static T JsDeserialize<T>(string json)
    {
        JavaScriptSerializer serializer = new JavaScriptSerializer();
        T o = serializer.Deserialize<T>(json);
        return o;
    }

    public static string Serialize<T>(T obj)
    {
        DataContractJsonSerializer serializer = new DataContractJsonSerializer(obj.GetType());

        MemoryStream ms = new MemoryStream();
        serializer.WriteObject(ms, obj);
        string retVal = Encoding.UTF8.GetString(ms.ToArray());
        return retVal;
    }

    public static T Deserialize<T>(string json)
    {
        T obj = Activator.CreateInstance<T>();
        MemoryStream ms = new MemoryStream(Encoding.Unicode.GetBytes(json));
        DataContractJsonSerializer serializer = new DataContractJsonSerializer(obj.GetType());
        obj = (T)serializer.ReadObject(ms);
        ms.Close();
        return obj;
    }

    protected void Button1_Click(object sender, EventArgs e)
    {
        JsonObject t = new JsonObject();
        t.ID = "c1";
        t.Text = "mmx";
        t.Expire = DateTime.Now.AddHours(-1);
        t.IsSomething = true;
        t.Cost = 5.1m;
        t.Scripts.Add(new Script("sss", true, ScriptTypes.Include));
        t.Scripts.Add(new Script("56", false, ScriptTypes.Startup));

        string j = Serialize<JsonObject>(t);
        //string j = Serialize<HttpRequest>(this.Request);

        JsonObject t2 = Deserialize<JsonObject>(j);
    }

    protected void Button4_Click(object sender, EventArgs e)
    {
        JsonObject t = new JsonObject();
        t.ID = "c1";
        t.Text = "mmx";
        t.Expire = DateTime.Now.AddHours(-1);
        t.IsSomething = true;
        t.Cost = 5.1m;
        t.Scripts.Add(new Script("sss", true, ScriptTypes.Include));
        t.Scripts.Add(new Script("56", false, ScriptTypes.Startup));

        string j = JsSerialize<JsonObject>(t);

        JsonObject t2 = JsDeserialize<JsonObject>(j);

    }
}

[Serializable()]
public class JsonObject
{
    public string ID { get; set; }
    public string Text { get; set; }
    public DateTime Expire { get; set; }
    public bool IsSomething { get; set; }
    public decimal Cost { get; set; }
    private ScriptCollection _Scripts = new ScriptCollection();
    public ScriptCollection Scripts { get { return _Scripts; } }
}

public enum ScriptTypes
{
    Include = 1,
    Block = 2,
    Startup = 3
}

[Serializable()]
public class Script
{
    public Script() { }
    public Script(string key, bool isValid, ScriptTypes type)
    {
        Key = key;
        IsValid = isValid;
        Type = type;
    }
    public string Key { get; set; }
    public bool IsValid { get; set; }
    public ScriptTypes Type { get; set; }
}

[Serializable()]
public class ScriptCollection : List<Script>
{
    public Script this[string Key]
    {
        get
        {
            var q = this.Where(x => x.Key == Key);
            return q.FirstOrDefault();
        }
    }
}

redpajama_set_name: RedPajamaGithub