text:
Talking about money doesn't have to make us cringe. Here's how to make it easier
How to approach conversations about money with confidence.
Therapists share tips for talking about money at work, with friends and in romantic relationships.TODAY illustration / Getty Images
Aug. 31, 2020, 9:14 PM UTC / Source: TODAY
By Chrissy Callahan
From a young age, we're taught to steer clear of three hot topics in conversations: religion, politics and money. And while the latter might seem like the least polarizing of the three subjects, especially these days, it's actually one that can make us feel the most uncomfortable.
Think about it: Over time, you've learned to dismiss your opinionated uncle's thoughts on politics and you're a master at brushing aside your parents' stern views on religion. But there's something about the topic of money that can really get under your skin. Maybe you're terrified of asking for a raise or don't know how to approach your partner about their spending habits without getting angry. Or perhaps you're not quite sure how to broach the subject of splitting the bill equitably with your group of friends.
Whatever your money conundrum is, you're not alone. TMRW reached out to several mental health experts to get tips for approaching some common financial conversations with confidence. With their help, you'll be mastering the art of money talk in no time!
Why does money talk make us cringe?
"People often don't talk about money because it is perceived to be socially disrespectful. There is a social implication and taboo around money that suggests we are more likely to talk about sex than money," said Debra L. Kaplan, therapist, financial expert and author of "Battle of the Titans: Mastering the Forces of Sex, Money, and Power in Relationships" and "For Love and Money: Exploring Sexual & Financial Betrayal in Relationships."
Our view of money is formed at a young age, and although everyone relates to the topic differently, many of us have a somewhat complicated relationship with it.
"For many, self-worth unfortunately becomes equated with financial worth, so there are often feelings of shame laced to feelings about money," said Stanley Teitelbaum, a financial therapist and clinical psychologist. "For some people, talking about money connects to their underlying fear of failure and/or fear of success, which relates to earlier versions of how they were perceived and treated by parents."
So it's not surprising that many of us keep our views about finances and our own financial status to ourselves.
5 money moves you should be making right now
"Discomfort with money talk stems from fear of judgement. For those with few resources, they fear being looked down upon. People may think they are lazy, stupid or a poor money manager. For those with lots of resources, they fear being exploited. People will not like them because of who they are, but for what they have," said John P. Vincent, a psychology professor at University of Houston.
Why we shy away from money talk at work — and how to advocate for yourself
Learning to negotiate your salary is a key skill that can pay off (quite literally) in the long run, but many of us find the negotiation part a bit intimidating. After all, when you're looking for a new job, a potential employer can always move on to a candidate who's willing to accept less money. Or if you're trying to score a raise in your current position, it's also possible that things could get awkward if your boss shuts you down.
"People's fear of talking about money at work or asking for a raise stems from fear of rejection or negative evaluation," Vincent said. "The potential emotional and real-life dangers of being turned down all contribute to avoidance."
Many times, a case of imposter syndrome can also make you second guess yourself and prevent you from attempting negotiation. "Some (fears when it comes to talking about money) goes back to a socially ingrained hesitation about feeling as though we are overstating our abilities, or bragging about what we feel we deserve," licensed professional counselor John A. Cooper said.
Hoping to get more comfortable with the topic of money at work? Try these techniques:
Practice what you're going to say: "I encourage clients to consider possible responses by their managers (both positive and negative), and explore their reactions to these different responses as they work to develop more comfort with the subject, Cooper said.
Remember, valuing your self-worth doesn't mean you're self-serving: "To ask about compensation, salary and income is about valuing one's self and our self-worth. We are advocating to be compensated for the contribution that we make," Kaplan said. "To ensure that a salary and compensation is commensurate with our training states that we are valuing our self. This is a profound statement of valuing self and not self-serving."
Keep your eyes on the prize: "It is important to remember what the goal of the conversation is: to connect, find information or begin a conversation to better understand another and be understood," Kaplan said.
Why we avoid money talk with our partners — and how to start the conversation
When you start dating someone new, the whole "reaching for the check" moment is just the beginning of many moments where you will have to tackle the topic of money. Naturally, discussing your credit score and spending habits isn't exactly the sexiest thing to do early on in a relationship, so many new couples avoid the topic for as long as possible.
"When considering a longer-term relationship, most people never ask questions like, 'What are your deal breakers for money and work?' for fear that they will be perceived by the other as presumptuous or greedy," Kaplan said. "But two people who are beginning a relationship and are starting to think about being together need to factor money into the conversation as much as sexual interest."
As time goes on, money talk can really make or break your relationship as you begin to learn how your partner approaches spending, saving, debt and more. And it's important to start the conversation sooner rather than later.
"It is vital to have the courage to share from a vulnerable standpoint and say 'I am hesitant and a bit embarrassed to have this conversation but I value you, us and what we have here. I would like to talk together about our beliefs and behaviors with money,'" Kaplan said.
Want to get more comfortable with the topic of money in your relationship? Try these techniques:
Develop a plan and stick to it: "To achieve a fiscally healthy relationship, start with identifying your goals and find a way to track how well actual expenses map onto those goals," Vincent said. "'Budget' is not a dirty word; it's just a way to be purposeful about your spending and saving behavior."
Avoid placing blame, but don't avoid healthy conflict: "As a relationship develops, many of us continue to struggle around dealing openly with money because it can easily become a source of conflict, leading to unspoken jealousy, resentment, distrust or frustration," Cooper said. "As with so many aspects of relationship building, developing a sense of cooperation rather than competition in the creation of a strong, healthy relationship can facilitate openness, transparency, and a feeling of working together rather than against each other."
Ask for a neutral party's help: "Some couples can find conversations about money fairly easy, while others can benefit from having a qualified financial therapist who can serve as a sort of buffer as they attempt to navigate the territory around this subject," Cooper said. "This is especially helpful if the partners in the relationship have conflicting 'money scripts' (developed patterns of thinking about money and how it impacts their lives, both individually and together)."
Why talking about money with friends is so hard — and how to make it easier
Whether you feel like you're catching up to pals who are seem to be more established in their careers or you're the one who's earning a lot more, talking about money between friends can often feel uncomfortable. After all, the topic of salaries can drudge up feelings of resentment and frustration, so many of us steer clear of it as much as possible.
"Sometimes for friends to talk about income is to consciously or unconsciously stir up a feeling of competition. 'If I make more, I must be more important or more valuable or have attained greater success in life, etc.,'" Cooper said.
But if you're regularly seeing your pals, the topic of money is bound to come up sooner or later, especially when you're trying to figure out how to split expenses during a night out at your favorite restaurant or organizing a bridal shower with your inner circle.
"Because the income disparity between friends can be significant, sometimes issues around splitting a dinner check 50-50 may affect one party more than another, even if it may seem equal on the surface," Cooper said. "For one member of the group to question such an arrangement, however, would run the risk of alerting the group that the questioning member is in fact more detrimentally affected, and thus could be seen as having a 'subordinate' financial position."
Want to get more comfortable talking about money with friends? Try these techniques:
Take the awkwardness out of that dinner check moment: "With friendships, the best strategy is preemptive. When going out to dinner, start off by saying 'Let's split it' or 'Separate checks' or 'Let's go Dutch,'" Vincent said. "Taking turns is also an option, as long as paying for an expensive steak dinner is balanced with buying pizza."
Avoid event-planning anxiety: "With events you plan with friends (like a bachelorette party), the first questions should be 'What's the budget?' and 'How much should each of us contribute?' That way you can determine if you can afford Dom Perignon or Bud Light," Vincent said.
Remember, it's awkward for your friends, too: "Often, when people open the subject of money by talking about their own conflicting feelings about the subject, rather than getting caught up in the numbers, other conversational partners can feel safer about jumping in with their own conflicts and insecurities," Cooper said.
Chrissy Callahan
Chrissy Callahan covers a range of topics for TODAY.com, including fashion, beauty, pop culture and food. In her free time, she enjoys traveling, watching bad reality TV and consuming copious amounts of cookie dough.


Guillaume Gouffier, Lord of Bonnivet, Admiral of France (c.1488-1525) was a French commander who performed well in battle early in his career, but who proved to be an ineffective commander when given control of the French army in Italy in 1523-24.
Bonnivet first came to prominence at the siege of Genoa of 1507 (fought during a gap in the Italian Wars to defeat a revolt against French dominance in Genoa). He was also present at the battle of Guinegate (16 August 1513), during the War of the Holy League, a French defeat that became known as the Battle of the Spurs because the French knights were said to have abandoned their spurs during their rapid retreat.
Bonnivet was a favourite of Francis I. He was made Admiral of France soon after Francis came to the throne in 1515 and took part in Francis's first invasion of Italy (1515-16). He was present at Francis's great victory at Marignano (13-14 September 1515), which established French control of Milan.
After the end of this war he served as a diplomat, first going to England in 1518 (where he helped arrange the planned marriage between Princess Mary (the future Mary I) and the Dauphin, and then serving as Francis's representative at the Imperial Diet at Frankfurt am Main of 1519, where he led Francis's efforts to be elected Holy Roman Emperor. Instead Charles I of Spain was elected as the Emperor Charles V, helping to trigger a long period of conflict between the two monarchs.
At the start of the First Hapsburg-Valois War (1521-26) Bonnivet captured Fontarabia (modern Fuenterrabia or Hondarribia, on the western bank of the Bidasoa River, at the western end of the Pyrenees), which fell in September 1521.
Francis I had intended to lead an army into Italy in person in 1523, but he alienated Charles, duke of Bourbon. The duke offered his services to Charles V, and although a plot to invade and divide France between Bourbon and the English never came to anything, Francis decided that he couldn't risk leaded the army in person. Instead Bonnivet was given the command.
Bonnivet crossed into Italy towards the end of the summer of 1523 and in September he captured Novara. He was then pinned down by Imperial forces under Prospero Colonna. He decided not to risk an assault on Milan, and instead prepared for a long blockade of the city. This lasted into December, before Bonnivet was forced to lift the siege and go into winter quarters on the Ticino.
By the spring of 1524 Colonna had died, and had been replaced by Charles de Lannoy, Viceroy of Naples. The Imperial forces caught the French in their winter quarters and forced them to retreat in some disorder. The French were forced to abandon their base at Abbiategrasso, not far to the west of Milan and retreat to Vigenavo, west of the Ticinio, then pulled back north-west to Novara. The French were expecting help from their Grison allies, but when the Grisons arrived at Chiavenna, north of Lake Como, they found no French to greet them and were forced back by Imperial forces under Giovanni de Medici. Bonnivet moved west again, this time to join up with his Swiss allies on the Sesia, but the army was now unwilling to stand and fight and Bonnivet was forced to order a retreat.
Bonnivet was wounded during the retreat. Command passed to the famous French leader Pierre Terrail, seigneur of Bayard, but he was killed while commanding the rear-guard at the battle of the Sesia (30 April 1524). Bonnivet was then replaced by Count Francois de St. Pol who led the army out of Italy.
The Imperial army followed the French out of Italy and began a siege of Marseille. Francis raised a relief army, which included Bonnivet, who had recovered from his wounds. The Admiral accompanied Francis's invasion of Italy of 1524, and was killed in the major French defeat at Pavia (24 February 1524).


text:
\section{INTRODUCTION}

Neutron stars are the densest stellar objects,  providing a unique laboratory for studying matter in physical conditions that cannot be replicated on Earth and are only found in these neutron-packed remnants of massive stars ($\gtrapprox 8 M_\odot$). Insights about the forms of matter which emerge under these extreme conditions can improve our understanding of two of the least well-understood fundamental forces, quantum chromodynamics, and gravity.

The neutron-rich matter within the inner regions of a neutron star can reach supranuclear densities of $10^{15} ~\text{g/cm}^3$, potentially leading to transitions to stable non-nucleonic states of strange matter in the form of hyperons~\cite{Tolos:2020aln,li2018competition, spinella2019hyperonic}, deconfined quark matter made of up, down, and strange quarks~\cite{Fukushima:2013rx,orsaria2014quark}, color superconducting phases~\cite{Alford:2007xm,zdunik2013maximum}, or  Bose-Einstein condensates made of negatively charged pions or $K^-$ mesons~\cite{baym1973pion,KAPLAN,ellis1995kaon}. The structure and behavior of matter at such extreme densities are one of the great mysteries in modern science, prompting decades of theoretical and experimental research into the interior composition of a neutron star.

The nature of matter within a neutron star is compactly summarized by its equation of state (EOS),  the relationship between pressure $P$ and energy density $\epsilon$, which show starkly different behaviors under the various states of strange matter hypotheses described above.
This relationship is determined by the microphysical interactions between various particles within the star, relative abundances of different particle species, as well as the star's temperature. Understanding the EOS of supranuclear density matter has been of interest to the nuclear and astrophysics communities for decades, resulting in many proposed phenomenological models for the equation of state of neutron star matter. Variations in models arise from a lack of precise knowledge of nuclear interactions between particles at such extreme conditions, as well as the wide range of densities and isospin asymmetries that are thought to exist within neutron stars \cite{1987ApJ...314..594F}.

While the internal pressure and density cannot be directly observed, the EOS of a star in static gravitational equilibrium determines stellar properties such as its mass and radius, which in turn determine observables such as the stellar X-ray spectrum.  Conversely, the stellar spectra can be used to infer the masses and radii, which in principle allow inference of the EOS~\cite{Rutledge,Heinke06,Lattimer01,PhysRevD.82.103011,Steiner10te,Lindblom:2013xkra}, though inversion of this second step is numerically very difficult. Additional challenges are due to the small number of neutron star observations, $\mathcal{O}(10)$, and the significant uncertainty of individual measurements.  It is therefore vital that as much information as possible is extracted from each star, and that the uncertainties be propagated accurately, to provide the most complete information possible about the EOS.

At the same time, there has been a dramatic burst of progress in artificial intelligence, specifically deep learning~\cite{baldi2021deep}, a modern re-branding of neural networks. This progress has led to breakthroughs not only in traditional areas such as natural language processing and computer vision but also in the natural sciences, including particle physics, often increasing the statistical power of difficult-to-collect data~\cite{baldi2014searching} while allowing robust handling of uncertainties~\cite{Ghosh:2021roe}.  Where earlier neural networks were limited in size, computing progress especially in the form of Graphical Processing Units (GPUs), has enabled the deployment of larger and  deeper networks that can handle more complex and higher-dimensional data~\cite{Baldi:2016fzo,Guest:2016iqz}, allowing direct analysis of data without requiring dimensional reduction, or other preprocessing steps, that can often sacrifice useful information.  The full power of these techniques has not yet been brought to bear on many astrophysical tasks.

In the context of the inference of neutron star EOS, recent work by Fujimoto {\it et al.}~\cite{Fujimoto:2019hxv,Fujimoto:2021zas} demonstrated the ability of deep networks to regress the EOS directly from a set of stellar mass-radius pairs, without the need to extract the functional relationship between mass and radius. Their analysis used a toy model to describe the uncertainties in mass and radius, assuming uncorrelated Gaussian errors randomly drawn from ad-hoc priors. Related work~\cite{Morawski:2020izm} has demonstrated similar regression, again assuming Gaussian uncertainty on mass and radius values, but with clever efforts to reduce dependence on EOS parameterization. An alternative approach~\cite{Ferreira:2019bny} uses both neural networks and support vector machines to regress the EOS from stellar radii and tidal deformations. What has received less attention are methods that directly access the observed telescope spectra, which underly the inferred stellar summary quantities such as mass and radius, and methods that allow for the full propagation of realistic uncertainties.

In this paper, we use neural network regression to infer the neutron star EOS from stellar mass and radius information derived from a realistic model of the underlying spectra and accounting for the associated underlying uncertainties. We subsequently introduce a network capable of directly analyzing high-dimensional neutron star spectra, performing inference of stellar mass and radii from telescope spectra as a demonstration of the capacity of deep networks.  Finally, we perform a first-of-its-kind inference of the EOS parameters directly from a multi-star {\it set} of stellar spectra, without requiring the intermediate step of collapsing the information into mass and radius; see Figure ~\ref{fig:scheme}.  In each case, we allow for full propagation of uncertainties by conditioning the networks on the stellar nuisance parameters. Comparisons are made to benchmark methods, where possible.

The paper is organized as follows. In Section~\ref{sec:bg}, we provide background on the physics of the connection between the nuclear equation of state and the stellar observations. Section~\ref{sec:ml} describes the fundamentals of the machine learning concepts on which our studies rely.  Details of the samples of simulated data are given in Section~\ref{sec:data}. Section~\ref{sec:eosmr} demonstrates inference of the EOS parameters from mass and radius, while Section~\ref{sec:mr} describes how mass and radius parameters can be inferred directly from stellar spectra, and Section~\ref{sec:e2e} shows end-to-end inference of EOS parameters from a set of neutron star spectra.

\section{Background}
\label{sec:bg}

\begin{figure*}
    \centering
    \includegraphics[width=0.85\textwidth]{figs/schematic_ns_ml.pdf}
    \caption{ Overview of the regression task, which involves either inferring stellar summary quantities such as mass and radius, which can then be used to deduce the equation of state as in earlier work~\cite{Fujimoto:2019hxv,Fujimoto:2021zas,Morawski:2020izm,Ferreira:2019bny} or inference of EOS directly from stellar spectra, as is demonstrated in this study.}
    \label{fig:scheme}
\end{figure*}

\subsection{Equation of State for Dense Matter}

Neutron star interiors present a unique opportunity to study matter under conditions beyond the reach of terrestrial laboratories: matter that is extremely high in density, relatively cold in temperature, and isospin-asymmetric \cite{lattimer}. Perhaps the closest experimental constraints have come from ultrarelativistic heavy-ion collisions (as conducted at the RHIC~\cite{muller2006results}, the LHC~\cite{aad2008atlas}, and FAIR~\cite{spiller2006fair}), which probe the nature of hot,
symmetric nuclear matter Extremely neutron-rich matter is more recently probed in studies of neutron skin or giant monopole and dipole resonances, but these studies are limited in nature \cite{nskin}. Unfortunately, these experiments currently lack the temperature and density constraints present within the core of a neutron star - meaning the dense matter encountered within a neutron star cannot yet be replicated by experiment.

While properties of superdense matter cannot be derived directly from experiments, certain general principles from general relativity and quantum chromodynamics (QCD) guide the theoretical investigation into neutron star matter. Neutron star structure is controlled by the long-range gravitational force~\cite{glendenning2012compact,RevModPhys.88.021001} which holds the star together and short-range strong interactions between nucleons and nuclei which provide the pressure that prevents the star from collapsing. At low nuclear densities (below nuclear saturation), effective field theories based on QCD provide a systematic basis for nuclear forces, which offers good constraints on two-nucleon interactions \cite{Machleidt:2011zz,RevModPhys.88.021001,qcd}. At higher densities, the QCD framework predicts that baryonic matter (where quarks are confined within hadrons) will experience a phase transition to quark-gluon plasma (QGP), where quarks and gluons are freed from hadronic boundaries \cite{qcd}. Other stable states of non-nucleonic matter may also occur, like the formation of hyperons, color superconducting phases of quark matter, or Bose-Einstein condensates of different mesons \cite{baym1973pion, ellis1995kaon}. Theoretical uncertainties have resulted in a wide range of proposed phenomenological models for the EOS of neutron star matter, which can then be tested by experiment or observation.

The EOS of neutron star matter is intrinsically linked to macroscopic characteristics like gravitational mass $M$ and radius $R$ through the general relativistic stellar structure equation known as the Tolman-Oppenheimer-Volkoff (TOV) equation \cite{PhysRev.55.374,PhysRev.55.364}. This equation assumes the object is spherically symmetric, non-rotating, and non-magnetic. The TOV equation is given by (assuming geometrized units where $G=c=1$):
\begin{equation} \label{eq:1}
    \frac{dP}{dr} = -\frac{(\epsilon+P)(m+4\pi r^3 P)}{r^2\left(1-\frac{2m}{r}\right)}
\end{equation}
where $m$ is the gravitational mass enclosed within a sphere of radius $r$. The mass of the star can be solved for as:
\begin{equation} \label{eq:2}
    \frac{dm}{dr} = 4\pi r^2 \epsilon
\end{equation}
where the total gravitational mass $M$ of a star with radius $R$ is given by $M \equiv m(R) = 4\pi \int_0^R dr ~r^2\epsilon$. Given an EOS, numerically solving the TOV equation for $M$ and $R$ is straightforward. These equations create a one-to-one map from the EOS to the $M-R$ relation \cite{Lindblom:2013xkra}; the inverse form of this map can therefore provide constraints on the EOS from observable properties. To mathematically invert the TOV equation, at least two stars' mass and radius must be known exactly, a feat is not possible with current observational technology. Solving the inverse problem is therefore much more complicated, potentially even intractable without making significant numerical assumptions.

\subsection{X-Ray Spectroscopy for Neutron Stars}

Many reliable observations of neutron stars come from X-ray emission, either from electromagnetic radiation from pulsars or thermal emission in quiescent low-mass X-ray binaries (qLMXBs). qLMXBs are particularly desirable to place strong constraints on neutron star structure as they are likely to have low magnetic fields (10$^{8-9}$ G), resulting in minimal effects on the radiation transport or temperature distribution on the star's surface \cite{2016ApJ...831..184B, campana1998neutron, potekhin2014atmospheres}. Additionally, these binaries are identified in globular clusters where distances, ages, and reddening are well-constrained \cite{Heinke_2003}. The distinctive soft thermal spectra from these sources come from a long-lived thermal glow resulting from heat stored in the deep crust of the neutron stars within the binary system during accretion, which is then re-radiated from the whole surface when accretion stops \cite{brown1998crustal}. For the context of this work, the inference of EOS will come from simulated thermal spectra from qLMXBs.

Observation of neutron star emission, whether X-ray or gravitational wave, has long served as a way to constrain mass and radius for neutron stars (eg. \cite{hebeler2013equation, Steiner18ct}), but uncertainties arise in the inference of these properties for a variety of reasons. In the case of X-ray radiation from qLMXBs, constraints on mass and radius are determined by fitting the emitted spectrum with an appropriate atmosphere model (where the surface composition is known or can be determined by the X-ray spectrum) and combining the spectroscopic measurements with the distance of the source. Models for thermal X-ray radiation are based on a light-element atmosphere, as the lightest element that is present in the atmosphere floats to the top due to rapid gravitational settling on neutron star surfaces \cite{2016ApJ...831..184B}. Atmospheric models used on X-ray spectra from qLMXBs gave the first broad constraints on neutron star radius, and more modern analyses of X-ray spectra have provided tighter constraints on both radius and EOS.

The high-resolution imaging and spectroscopy of NASA's \textit{Chandra X-ray Observatory} have provided powerful insight into neutron star properties like cooling \cite{cooling, wijnands2017cooling}, mass and radius \cite{2016ApJ...831..184B}, and binary mergers of exotic stars \cite{mag}. \textit{Chandra}'s telescope contains a system of four pairs of mirrors that focus incoming X-ray photons to the Advanced CCD Imaging Spectrometer (ACIS), which measures the energy of each incoming X-ray. The observed spectrum, along with a corresponding instrument response, is then fit to a well-motivated parameterized model. Many such models for spectral fitting exist in {\sc xspec}~\cite{xspec}, an X-ray spectral fitting package distributed and maintained by the aegis of the GSFC High Energy Astrophysics Science Archival Research Center (HEASARC). {\sc xspec}\ has been used numerous times in the past to analyze data from \textit{Chandra} as well as other spectrometers like \textit{NICER}, \textit{Nustar}, and \textit{XMM-Newton}, making it a valuable resource for inference of neutron star properties.

\section{Machine Learning}
\label{sec:ml}

Machine learning methods, in particular deep learning, aim to extract useful knowledge from data automatically and are rapidly being applied across many data-rich fields of science~\cite{baldi2021deep}. In regression tasks such as EOS inference, one is interested in constructing a function $f$ whose inputs are the observed data and whose outputs are an estimate of some parameter of interest.  The optimal function $f$ is not known initially, but an approximation can be learned from a set of input-output example pairs.

In order to approximate $f$, machine learning methods first begin with a suitable class $C$ of parameterized functions (e.g. polynomials of a certain degree, neural networks of a certain architecture) and then seek to find the best approximation to $f$ within
the class $C$. This is typically done through a stochastic gradient descent procedure that seeks to iteratively minimize the approximation error on the training set.

The well-known technique of linear regression is the most elementary form of regression, and can be viewed as a form of shallow learning (no hidden layers). Deep learning generalizes linear regression by using multi-layer neural networks as the class $C$ and thus enabling the construction of sophisticated and flexible non-linear approximations.
With sufficient training data and computing power, deep learning methods can handle large-scale problems with high dimensional data and avoid heuristic simplifications that lose information. It is not uncommon to deal with problems with input sizes in the range of up to $10^9$ examples, each with dimensions of $10^{3-4}$, with neural networks that can have up to $10^{11}$ free parameters. Training sets can range in size from $10^1$ to $10^{10}$ or more.  Unlike shallow learning and linear regression,  deep learning does not require that the number of parameters be equal to the number of training examples~\cite{neyshabur2017exploring}. More recent, attention-based architectures, such as transformers~\cite{transformers,quarks2022baldi}, allow networks to take advantage of structures and symmetries in the data, and are applied in sections below.

When the interpretation of data depends on external unmeasured or poorly-known parameters, such as neutron star temperature or distance, it can be useful to apply  {\it parameterized networks}~\cite{Baldi:2016fzo}. Such networks learn a task as a function of the external parameter, allowing for evaluation of a fixed dataset under varying assumptions about the parameter~\cite{Ghosh:2021roe}.

\iffalse
\begin{itemize}
    \item general intro to machine learning: what can it do, what does it need - this should start pretty basic as ML is not used readily in astro
    \begin{itemize}
        \item ML encompasses a variety of algorithms and modeling techniques that have many uses - recently the application to scientific problems
        \item ML can handle large scale problems with inputs hundreds to millions in dimensionality (both in input data and number of adjustable parameters)
    \end{itemize}
    \item ML can handle problems where parametric functions are not flexible enough
    \item ML can handle higher-dim data and avoid heuristic simplifications that lose information
    \item types of ML we use in this paper: NNs conditioned on a parameter~\cite{Baldi:2016fzo}
\end{itemize}
\fi

\section{TRAINING SAMPLES}
\label{sec:data}

Samples of simulated neutron stars, used to train networks and evaluate their performance, are described below.

Each simulated star is described by two high-level summary quantities, mass and radius, which are drawn from the mass-radius relation determined by the EOS, as well as three nuisance parameters that are independent of the EOS and can vary from star to star.  These five parameters are sufficient to determine the expected simulated Chandra telescope spectrum in the chosen NS theoretical model.  In the case of EOS inference, sets of stars with consistent EOS are grouped to form training and testing sets. Details of each step of the generation are provided below.

\subsection{Equation of State}

The equation of state of the hadronic matter within the core is modeled with the relativistic non-linear mean field model GM1L~\cite{Typel}. The version used here only accounts for protons and neutrons but can be extended to include hyperons and $\Delta$ baryons~\cite{Spinella:2020WSBook}. The corresponding saturation properties of symmetric nuclear matter for the GM1L parametrization are shown in Table \ref{tab:gm1l} \cite{Spinella:2020WSBook,Malfatti19}. These properties include the nuclear saturation density $n_0$, energy per nucleon $E_0$, nuclear compressibility $K_0$, effective nucleon mass $m^*_N/m_N$, asymmetry energy $J$, asymmetry energy slope $L_0$, and the value of the nucleon potential $U_N$. The values of $L_0$ listed in Table \ref{tab:gm1l} are in agreement with the value of the slope of the symmetry energy deduced from nuclear experiments and astrophysical observations \cite{Oertel}.

The most commonly used constraints on $K_0$ come from  experimental values of the giant monopole resonance, which lie in the range of 220 to 260 MeV \cite{Shlomo2006,Garg_2018}. The analysis of \cite{Stone2014}, however, suggests a higher range of 250 to 315 MeV. The value of $K_0=300$~MeV considered in our paper falls into the latter category, but this will not dramatically impact the neutron-rich equation of state appropriate for neutron star interiors. The GM1L equation of state for the core is paired with two models for the crust. For the outer crust, which falls in the density range $10^4-10^{11} ~\text{g/cm}^3$, we use the Baym-Pethick-Sutherland (BPS) model \cite{1971ApJ...170..299B}. For the inner crust, with densities in the range $10^{11}-10^{14} ~\text{g/cm}^3$, we use the Baym-Bethe-Pethick (BBP) model \cite{BAYM1971225}.

\begin{table}[]
    \caption{Parameters of the model used to select example equations of state for the generation of simulated data samples. Shown are properties of the  symmetric nuclear matter at saturation density for the GM1L parametrization of neutron star interiors\cite{Typel}; see text for details.}
    \label{tab:gm1l}
    \centering
    \begin{tabular}{ccc}
             \hline \hline
         Saturation Property ~~~~& Value ~~~~&  Units \\
                  \hline
        $n_0$ & 0.153 & fm$^{-3}$ \\
        $E_0$ & -16.3 & MeV \\
        $K_0$ & 300.0 & MeV \\
        $m^*_N/m_N$ & 0.70 & - \\
        $J$ & 32.5 & MeV \\
        $L_0$ & 55.0 & MeV \\
        $U_N$ & -65.5 & MeV \\
         \hline \hline
    \end{tabular}
\end{table}

To limit the number of parameters the networks must learn, the essential features of the high-density portion of the EOS needed to be represented efficiently by just a few values. This can be done accurately by constructing parametric representations based on spectral fits, formed as generalizations of the Fourier series used to represent periodic functions \cite{PhysRevD.82.103011}. An EOS, defined as $P = P(\epsilon)$ or $\epsilon(P) = \epsilon$, can be represented as a linear combination of basis functions $\epsilon_k(\phi)$:
\begin{equation} \label{eq:ep}
    \epsilon(p) = \sum_{k}\epsilon_k \Phi_k(p)
\end{equation}

\noindent where $\phi_k(p)$ can be any complete set of functions. The EOS is therefore determined by the spectral coefficients $\epsilon_k$, making $ \epsilon_k = \epsilon(p,\epsilon_k)$. There are two important conditions that a physical EOS must satisfy to ensure microscopic stability. The first is that the EOS must be non-negative, or $p(\epsilon) \geq  0$, and the second is that pressure must be monotonically increasing with density \cite{PhysRevD.82.103011}. Because these conditions are not naturally respected by arbitrary basis functions in a spectral representation, representing an EOS with a straightforward spectral expansion will likely produce data that violates microscopic stability and is therefore erroneous. To ensure these two conditions are met, we instead turn to a faithful construction of spectral representations of the EOS; the process for constructing these is outlined in detail in \cite{PhysRevD.97.123019} and \cite{PhysRevD.82.103011}.

The spectral representation of GM1L is formed from representing the EOS in terms of the relativistic enthalpy, $h$, where the EOS can be rewritten as a pair of equations $P = P(h)$ and $\epsilon = \epsilon(h)$. The enthalpy can be defined as
\begin{equation} \label{eq:hp}
    h(P) = \int_{0}^{P} \frac{dP'}{\epsilon(P')c^2+P'}
\end{equation}
where $c$ is the speed of light \cite{PhysRevD.97.123019}. Inverting Eq.~\ref{eq:hp} obtains the equation $P = P(h)$, which can recover the EOS $\epsilon(P)$ as $\epsilon(h) = \epsilon[P(h)]$. The pair of equations $P = P(h)$ and $\epsilon = \epsilon(h)$ can be expressed from a reduction to quadrature:
\begin{equation}
    P(h) = P_0 + (\epsilon_0c^2+P_0) \int_{h_0}^{h} \mu(h')dh',
\end{equation}
\begin{equation}
    \epsilon(h) = -P(h)c^{-2}+(\epsilon_0+p_0c^{-2}) \mu(h).
\end{equation}

The function $\mu(h)$ is defined as
\begin{equation}
    \mu(h) = \exp \left \{ \int_{h_0}^{h} [2+\Gamma(h')]dh'  \right \}
\end{equation}
which is dependent on the sound speed or velocity function $\Gamma(h)$ and constants $P_0 = P(h_0)$ and $\epsilon_0 = \epsilon(h_0)$ \cite{PhysRevD.97.123019}. Similar to Eq.~\ref{eq:ep}, the velocity function can be represented as a spectral expansion:
\begin{equation}
    \Gamma(h,\nu_k) = \exp \left[  \sum_{k} \nu_k\Phi_k(h) \right]
\end{equation}
where $\Phi_k(h)$ is any complete set of basis functions on the domain [$h_0,h_{max}$].

We constructed enthalpy-based causal spectral fits for the GM1L EOS with up to 10 parameters. Figure \ref{fig:sp_err} shows that just two spectral parameters produce parameterizations with a mean relative error of only 10\%, and additional parameters can reduce the error to 5\% or lower.  A small number of parameters is preferred due to the increased complexity of learning multiple parameters, and the danger of  Runge's phenomenon when applying our networks to current neutron star observations with accurate readings of mass and radius, which are still relatively few. Runge's phenomenon arises when attempting to fit equispaced data points with polynomials of high degree; increasing the order of the polynomial interpolation can result in issues with convergence or divergence rates for certain functions \cite{runge_phen}. When applied to neutron star observation, attempting to fit a small number of data points (with varying accuracy in observation method) with a model having many parameters may result in fits that accurately fit the data, but are very poor representations of the actual physics. Based on the reasons listed above, we chose to use two spectral parameters to represent the EOS, hereafter referred to as $\lambda_1$ and $\lambda_2$.

\begin{figure}[hbt!]
    \centering
        \includegraphics[width=0.3\textwidth]{figs/avg_sp_err.png}
    \caption{ Relative error in spectral parameterization of the equation of state, shown as a function of the number of parameters used.}
    \label{fig:sp_err}
\end{figure}

To create many samples needed for training and testing, spectral parameters were then constructed from the expression:
\[\lambda_{\text{generated}} = \lambda_{\text{true}} \cdot(1+2 \cdot\mathit{scale} (-0.5+\mathit{ran2}))\]
where $\lambda_{\text{generated}}$ represents the newly constructed spectral parameter, $\lambda_{\text{true}}$ is the best fit (true) spectral parameter of GM1L, and $\mathit{scale}$ is a scaling parameter set to 0.05.  $\mathit{ran2}$ are uniformly distributed random numbers in the range 0 to 1 generated by the $\mathit{ran2}$ function given in \cite{Press2007}. This process was repeated to create $10^4$ different EOS variations. Each EOS variation was used to generate a coinciding $M-R$ relation using equations \ref{eq:1} \ref{eq:2}, examples of which are seen in Figure ~\ref{fig:tdata_mr}, from which 100 $(M,R)$ pairs are selected, each representing stellar parameters consistent with that EOS. Due to the random component of our EOS generation, some models have a mass peak below the current observed mass limit, 2.1 $M_\odot$. All models have a minimum mass of at least 1 $M_\odot$. The physicality of predicted results will be discussed in further detail \ref{sec:Discussion}.

\begin{figure}[hbt!]
    \centering
        \includegraphics[width=0.42\textwidth]{figs/eos.pdf}
    \includegraphics[width=0.42\textwidth]{figs/mr.pdf}
    \caption{Examples of training data. Top, 100  samples in EOS spectral parameter space $(\lambda_1,\lambda_2)$ randomly selected from the full set of 10,000 EOS spectral pairs. Bottom, neutron star mass-radius curves determined by the selected EOS parameters.}

    \label{fig:tdata_mr}
\end{figure}

\subsection{Modeling X-ray Spectra}

The relation between stellar parameters $(M,R)$ is determined by the EOS, and samples from the allowed curve are used as input to generate simulated X-ray Chandra spectra, such as the Chandra observation of the quiescent low-mass X-ray binary (qLMXB) X7 in the globular cluster 47 Tuc~\cite{2016ApJ...831..184B}.

The {\sc xspec}\ program~\cite{xspec}, which can be used for spectral fitting, is also capable of generation of simulated spectra, via the  \texttt{fakeit} command when a NS model and telescope response matrix are provided.

The NS theoretical model \texttt{NSATMOS}~\cite{Heinke_2006} selected includes a hydrogen atmosphere model with electron conduction and self-irradiation. The Chandra telescope response specified in Ref.~\cite{Heinke_2006} was also used to describe the instrument response and telescope effective area.

\begin{figure}[hbt!]
    \centering
    \includegraphics[width=0.29\textwidth]{figs/scan_0.pdf}
    \includegraphics[width=0.29\textwidth]{figs/scan_1.pdf}
    \includegraphics[width=0.29\textwidth]{figs/scan_2.pdf}
    \includegraphics[width=0.29\textwidth]{figs/scan_3.pdf}
    \includegraphics[width=0.29\textwidth]{figs/scan_4.pdf}
    \caption{ Examples of simulated stellar spectra expected for several values of stellar parameters.  Each pane shows the expected rate of photons in Chandra per energy bin, for variations of the parameters of interest (mass M, radius R) as well as for variations of the nuisance parameters ($N_H$, log($T_{\textrm{eff}}$), distance). The dashed black line has the same parameters in each pane.}

    \label{fig:tdata_spectra}
\end{figure}

\subsection{Nuisance Parameters}

The \texttt{NSATMOS} model has five parameters to describe each star:
 gravitational mass $M$ in units of $M\odot$, radius $R$ in units of km, and three additional parameters related to observation. For the context of $M-R$ and subsequent EOS inference, only $M$ and $R$ are parameters of interest, whose values come from those generated by the GM1L EOS and so provide information relevant to the physical question. The remaining three nuisance parameters are the effective temperature of the surface, $T_{\mathrm{eff}}$, the distance to the star, $d$, and the hydrogen column, $N_H$ which parameterizes the reddening of the spectrum by the interstellar medium.  These parameters influence the observed spectrum of a given neutron star. Lack of knowledge of these values is a leading source of uncertainty in the inference of mass and radius, and hence EOS.

 Using Table 1 in Ref.~\cite{Steiner18ct} as a guide, we find that distances typically range between 2 and 10 kpc, and hydrogen columns lie between 0.2 and $5 \times 10^{21}~\mathrm{cm}^{-2}$. While neutron stars with larger distances and larger hydrogen columns exist, they are sufficiently distant as to be difficult to obtain spectral information. From Table 3 in Ref.~\cite{Lattimer14ns}, effective temperatures at the surface typically lie between 50 and 200 eV, or from $6 \times 10^5$ and $2.4 \times 10^6$ K. Note that core temperatures are typically a few orders of magnitude larger. Again colder neutron stars most certainly exist but are more difficult to observe.

Examples of generated spectra for varying stellar parameters are shown in Figure ~\ref{fig:tdata_spectra}. The generated spectra are very sensitive to the effective surface temperature, with lesser sensitivity to other parameters.

The networks detailed below provide estimates of either the neutron star mass and radius or the EOS parameters, conditioned on NP values. Uncertainty in regressed parameters of interest due to uncertainties in the NP can then be fully propagated via variation of the NPs used during regression.  To demonstrate the impact of NP uncertainties, we define three example scenarios of uncertainties, dubbed ``true", ``tight", and ``loose", which describe the quality of prior information on the NP values for each star.

     In the ``true" scenario, the NPs are set to the true value used to generate the spectra, such that the NP prior is essentially a delta function. In the ``tight" scenario, the uncertainty is described as a narrow  Gaussian for each NP, with distance having a width of 5\%, hydrogen column having a width of 30\%, and $\log(T_\textrm{eff})$ having a width of 0.1.
  In the ``loose" scenario, the uncertainties are described by a wider Gaussian, with distance having a width of 20\%, hydrogen column having a width of 50\%, and $\log(T_\textrm{eff})$ having a width of 0.2. These ranges are shown in Table \ref{tab:nps}. The sensitivity to NP values is reflected in the performance of the networks below.

\begin{table}[]
    \caption{Description of ``true", ``tight", and ``loose" nuisance parameter (NP) scenarios. Shown are the width of each Gaussian distribution representing the prior knowledge of each NP. For distance and $N_H$, width is relative; for log($T_{\text{eff}}$), it is absolute. See text for details and references.}
    \label{tab:nps}
    \centering
    \begin{tabular}{lcrrr}
             \hline \hline
         Nuis. Param. & True & Tight & Loose \\
                  \hline
        Distance & exact  & 5\% & 20\% \\
        Hydrogen Column $N_H$ &exact & 30\% & 50\% \\
        log($T_{\text{eff}}) $ &exact    & $\pm$0.1  & $\pm$0.2  \\
         \hline \hline
    \end{tabular}
\end{table}

\section{Inference of EOS from Mass and Radius}
\label{sec:eosmr}

Previous applications of machine learning to the task of inferring the equation of state have begun from the stellar mass and radii~\cite{Fujimoto:2019hxv,Fujimoto:2021zas,Morawski:2020izm}, or equivalent parameters~\cite{Ferreira:2019bny}, though with simple ad-hoc descriptions of the uncertainty on stellar mass and radius values, often modeled as two-dimensional uncorrelated Gaussian's rather than propagating the underlying uncertainties. In this section, we tackle the same problem, but where the stellar data are more realistic and the underlying uncertainties are fully propagated to the EOS estimation.  Specifically, the best estimates of stellar mass and radius are derived using state-of-the-art tools that extract them from realistic stellar spectra, which include the impact of stellar nuisance parameters and limited observation time. In addition, this mass-radius estimation is conditioned on the nuisance parameters, such that variations in those nuisance parameters lead to variations in the mass and radius estimates.  This connects directly to the neural network regression of EOS parameters from mass-radius values parameterized in the nuisance parameters, allowing for the direct propagation of the underlying uncertainties to give a measure of the resulting uncertainty on the regressed EOS parameters.

Below, we describe the extraction of realistic mass and radius values with {\sc xspec}\, and their subsequent use in NN regression of the EOS parameters and the estimation of the uncertainty.  For comparison, we also provide a demonstration of the regression of EOS parameters using polynomial regression. In subsequent sections, we consider an alternative extraction of mass and radius using a NN, as well as end-to-end regression of EOS directly from stellar spectra.

\subsection{Mass and Radius inference by XSPEC}

Sample stellar spectra are generated as described above, including Poisson noise corresponding to an observation time of 100 ks, and nuisance parameters variations as specified in Table ~\ref{tab:nps}.

Given a sample observed X-ray spectrum, the {\sc xspec}\  code scans the mass and radius parameter space, searching for values that best describe it. For each mass-radius pair, the expected spectrum is calculated using the chosen model and telescope response function,  identical to those used to generate the sample spectra being fit.  The fitted values are those which minimize a bin-wise $\chi^2$, and reported errors are those which generate a fixed increase in the $\chi^2$ metric.

To propagate the uncertainty due to the lack of knowledge of the NP values, the fit on a given spectrum is performed several times with varying assumed values of the NPs drawn from the appropriate prior. The variation in the resulting fitted values then describes the impact of uncertainty on the NPs.  For this reason, during each {\sc xspec}\ fit, the NP values are not allowed to vary, but are frozen.  Figure~\ref{fig:xspec_mr_np} shows examples for individual stars, demonstrating the variation of the stellar parameter estimates with varying NP values. For illustrative comparison to ad-hoc models of uncertainty, the standard deviation in mass and radius are used to define the widths of a 2d error ellipse, though it is clear that this fails to capture the complex nature of the impacts of the underlying uncertainties; these simple error models are not used in our analysis.

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{figs/xspec/loose2.pdf}
        \includegraphics[width=0.35\textwidth]{figs/xspec/loose5.pdf}
 \includegraphics[width=0.35\textwidth]{figs/xspec/tight2.pdf}
        \includegraphics[width=0.35\textwidth]{figs/xspec/tight5.pdf}
    \caption{ Estimation of the mass and radius of a neutron star from the underlying stellar spectra, by {\sc xspec}. Each pane represents one star, and shown (green) are estimates for several independent values of the nuisance parameters drawn from the associated priors, and the mean value (red). Top two cases have loose priors, bottom two have tight. The dashed ellipse, whose widths are set to the standard deviation of the mass and radius estimates, is a demonstration of the inadequacy of a simple uncertainty model.}
    \label{fig:xspec_mr_np}
\end{figure}

{\sc xspec}\  is also capable of {\it floating} the nuisance parameters, varying their values to improve an individual fit, and reporting an uncertainty envelope in the mass-radius plane. This can be helpful in the case where the mass and radius and their envelope are the final targets. However, to propagate the uncertainty downstream requires that we have the full posterior in the mass-radius plane or samples from it. An estimate and envelope do not provide that capacity, though they can allow for ad-hoc parameterizations of the prior as have been performed previously. We condition on the nuisance parameters to allow full propagation of the NP uncertainty through to EOS estimation, as we do below.

Performance of {\sc xspec}\  regression of mass and radius is shown in Figure ~\ref{fig:xspec_mr}, where the residuals increase as expected with wider priors on the nuisance parameters.  In addition, note that in the case of the ``loose'' priors, there is a small fraction of cases, where {\sc xspec}\ fails to converge on an estimate, as the nuisance parameters are fixed to a value far from the value used to generate the spectrum.

\begin{figure}
    \centering
   \includegraphics[scale=0.4]{figs/res_xspec_mr_mass.pdf}
   \includegraphics[scale=0.4]{figs/res_xspec_mr_radius.pdf}
    \caption{ Performance of {\sc xspec}\ inference of neutron star mass and radius, as measured by residuals between the fitted ("pred") and true values under three treatments of the nuisance parameters (NPs). In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively; see text for details. Cases in which {\sc xspec}\  fails to converge are shown as large negative residuals.}
    \label{fig:xspec_mr}
\end{figure}

\subsection{Inference of EOS from mass and radius}

In this section, we demonstrate the inference of EOS parameters from the stellar mass and radius data, using neural network regression parameterized in the NPs to allow propagation of the uncertainty. In addition, we build polynomial regression models which serve as a benchmark, following Ref.~\cite{Fujimoto:2019hxv}.

\subsubsection{Neural Network regression}
Deep feed-forward neural networks are trained to provide the EOS parameters given a collection of ten stars, each represented by their mass and radius. Each network has two outputs, $\lambda_1$ and $\lambda_2$.

We train three networks, one for each of the true, tight, and loose NP scenarios.
All networks have identical architecture, 10 hidden layers with 32 nodes each followed by an output layer with 2 nodes. Rectified linear units are used as activation functions for the hidden layers while linear activations are used for the output layers. They were trained up to 1000 epochs with a mean squared error loss and an Adam optimizer~\cite{adam}, and the performance is evaluated on independent validation data. The performance was not found to be highly sensitive to hyper-parameter tuning, so permutation symmetry preserving architectures were not explored. The networks were implemented using \texttt{Tensorflow 2.7.0} on a single \texttt{NVIDIA RTX A5000} GPU.

\subsubsection{Polynomial regression}

Following the example of Ref \cite{Fujimoto:2019hxv}, as a performance benchmark we also construct a polynomial regression model to regress EOS parameters $\lambda_1$ and $\lambda_2$ from stellar mass and radius information. The input for each model is a 20 x 1 vector containing the following information from each of ten stars: mass $M$, radius $R$.

Each 20 x 1 input to the polynomial regression network represents a set of stars, all of which are chosen from the same EOS. We construct two multivariate polynomial regression models of degree two, following the form:
\[\lambda_i = \beta_0 + \sum_{i=1,2}^N \beta_i x^i_1\]
where each $\beta$ are coefficients and $N = 20$ to represent the cluster size, or 10 mass-radius pairs. Polynomial features were created using the machine learning toolbox scikit-learn~\cite{scikit-learn}, and subsequently fit to a linear regression model. This model uses optimization in the form of ordinary least squares, which takes the form:
\[\text{min}_\beta ||X\beta - y||^2_2\]
where $X$ is the $N = 20$ input vector, and $y$ is the target EOS parameter, either $\lambda_1$ or $\lambda_2$.

\subsubsection{Estimation of Uncertainty}

The uncertainty in the underlying NS nuisance parameters has a significant impact on the estimation of EOS parameters. Because the mass and radius estimation is conditioned on the NPs, leading to variations in the mass and radius, (eg see Figure ~\ref{fig:xspec_mr_np}), those variations can be propagated through the EOS estimation. The significant uncertainties in stellar nuisance parameters and the small number of stars observed to date make the treatment of those uncertainties vital. The validity of the final result is only as powerful as the validity of its uncertainties. In Figure ~\ref{fig:xspec_eos_nps}, examples of the variation of the EOS estimates are shown, where the underlying stellar spectra are fixed. Thus, this provides a measure of the uncertainty in the EOS due to the uncertainty in the NPs.

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{figs/nnxspec/example_eos_NN_Loose0.pdf}
        \includegraphics[width=0.35\textwidth]{figs/nnxspec/example_eos_NN_Loose1.pdf}
    \includegraphics[width=0.35\textwidth]{figs/nnxspec/example_eos_NN_Tight0.pdf}
        \includegraphics[width=0.35\textwidth]{figs/nnxspec/example_eos_NN_Tight1.pdf}
    \caption{Neural network regression of the EOS parameters $\lambda_1$ and $\lambda_2$ of a set of 10 neutron stars from from their  masses and radii as estimated by {\sc xspec}\ from each stars spectrum. Each pane represents an example dataset of 10 simulated stars, and shown (green) are EOS estimates for several independent values of the stellar nuisance parameters drawn from the associated priors, and the mean value (red). Top two cases have loose priors, bottom two have tight.}
    \label{fig:xspec_eos_nps}
\end{figure}

\subsubsection{Performance}

Performance of neural network regression of the EOS parameters $\lambda_1$ and $\lambda_2$ are compared to polynomial regression of the same quantities using identical datasets via comparison of the residuals, the difference between the true and regressed values. As seen in Figures~\ref{fig:mr_to_eos1} and \ref{fig:mr_to_eos2}, while PR is able to achieve narrower residuals in the true case,
the network regression is  more robust in cases with larger uncertainties.

This result confirms what has been seen in earlier studies of NN regression from mass-radius pairs~\cite{Fujimoto:2019hxv}, but our study extends previous work by using realistic values of the mass and radius inferred from realistic simulated spectra, as well as by demonstrating uncertainty quantification, via full propagation of the underlying uncertainties due to nuisance parameters.

\begin{figure}[hbt!]
    \centering
   \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_true1.pdf}
   \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_tight1.pdf}
    \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_loose1.pdf}

    \caption{ Comparison of the performance of NN regression and polynomial regression of EOS parameter $\lambda_1$ from mass radius pairs inferred by {\sc xspec}\  from stellar spectra. Shown is the residual, the difference between the predicted and true values for each of three treatments of the stellar nuisance parameters. In the "true" case, the NPs are fixed to their true values; in the "tight" and "loose" cases, they are drawn from narrow or wide priors, respectively; see text for details. }
    \label{fig:mr_to_eos1}
\end{figure}

\begin{figure}[hbt!]
    \centering
    \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_true2.pdf}
   \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_tight2.pdf}
   \includegraphics[scale=0.3]{figs/res_nn_pr_xpsec_loose2.pdf}
    \caption{ Comparison of the performance of NN regression and polynomial regression of EOS parameter $\lambda_2$ from mass radius pairs inferred by {\sc xspec}\  from stellar spectra. Shown is the residual, the difference between the predicted and true values for each of three treatments of the stellar nuisance parameters. In the "true" case, the NPs are fixed to their true values; in the "tight" and "loose" cases, they are drawn from narrow or wide priors, respectively; see text for details. }
    \label{fig:mr_to_eos2}
\end{figure}

\FloatBarrier

\section{Inference of Mass and Radius from Spectra}
\label{sec:mr}

Previous applications of machine learning to neutron star datasets focus on analysis of mass-radius pairs, as demonstrated above, rather than direct analysis of the stellar spectra by neural networks.

A potential obstacle to direct analysis of spectra by neural networks is that the spectra are high-dimensional, often with $\mathcal{O}(10^3)$ bins of photon energy.
However deep learning methods combined with GPUs have no trouble analyzing data with similarly high dimensionality~\cite{Baldi:2016fzo}, opening up new opportunities to tackle this important topic.  While the mass and radius are powerful summaries of the information in the lower-level spectra which is relevant to the equation of state, direct ML analysis of the spectra themselves may allow for the extraction of additional information or provide more robust propagation of uncertainties.  As an initial step, we begin by estimating the stellar mass and radius from a single stellar spectrum before moving on to end-to-end inference of EOS parameters directly from a set of spectra in the next section.

In this section, we apply machine learning to the task of extracting the mass and radius from the stellar spectra, training a network we refer to as MR\_Net~. This serves as a demonstration of the capabilities of ML to grapple with high-dimensional datasets, allows us to harmonize the treatment of nuisance parameters end-to-end from spectra to EOS, and potentially extract more relevant information.

\subsection{MR\_Net~ Method}

We build a network whose inputs correspond to the bins of the stellar X-ray spectrum, and whose outputs are the estimates of the star's mass and radius.  In addition, the mass and radius regressor is parameterized on the stellar nuisance parameters (distance, $N_H$, $\log(T_\textrm{eff})$), which allows the results to be conditioned on the nuisance parameters.

This architecture is composed of two input branches, one to process the star's spectra and another to process the corresponding nuisance parameters. Each branch contains a series of layers that process its inputs in isolation. Following these initial layers, the output from the branches is combined, forming a single vector containing all the information. This vector is then passed to a final series of layers to predict the star's mass and radius. Each segment of the network, both the branches and the main trunk of the network, contain four layers, giving the network eight layers in total. All fully connected layers contain 275 nodes and utilize a dropout probability of 0.25. The network employs skip connections between alternate layers. This stabilizes the training process and adds robustness to the network overall. The network is trained using the Adam optimizer with an initial learning rate of 0.00017 which is slowly decayed over the course of training.

Mass-radius regression is formulated as a supervised learning problem, where the network learns to minimize the error between the true mass and radius and its predictions. The network weights are updated by stochastic  gradient descent using backpropagation. The Huber loss function is used and the Adam optimizer computes gradients and schedules the backward passes.

\subsection{MR\_Net~ Performance in Mass, Radius}

We begin with the best-case scenario in which the nuisance parameters are known with zero uncertainty, referred to as ``true NP" above.
Figure~\ref{fig:regressMR} shows the performance of MR\_Net~ given neutron star spectra with statistical noise corresponding to 100,000 seconds (100 ks) of observation time, as well as for spectra without statistical noise.  This demonstrates the contribution of statistical noise to the residual and demonstrates the network's ability to digest the spectral information and understand the dependence on mass and radius.

\begin{figure}[hbt!]
    \centering
    \includegraphics[scale=0.3]{figs/res_spectratoMR_mass.pdf}
    \includegraphics[scale=0.3]{figs/res_spectratoMR_radius}
    \caption{ Performance of the MR\_Net~ regression of a neutron star mass (top) and radius (bottom) from its stellar X-ray spectrum. Shown is the residual, the difference between the true and predicted values, for spectra with statistical noise corresponding to an observation time of 100ks, and for spectra without statistical noise, which demonstrates the capacity of the network. Nuisance parameters are fixed to their true values.}
    \label{fig:regressMR}
\end{figure}

What is clear is that MR\_Net~ is capable of extracting the mass and radius values of the star directly from the spectrum. We emphasize that the network is trained on examples, but does not benefit from the knowledge of the theoretical model used to generate these stars, while {\sc xspec}\  requires precise specification of the theoretical model.   This lack of requirement of a theoretical model opens new possibilities, such as training MR\_Net~ to interpolate smoothly between theoretical models, by providing a mixed or parameterized training set.

\begin{figure}[hbt!]
    \centering
    \includegraphics[scale=0.3]{figs/res_MRcompare_mass_true.pdf}
    \includegraphics[scale=0.3]{figs/res_MRcompare_mass_tight.pdf}
    \includegraphics[scale=0.3]{figs/res_MRcompare_mass_loose.pdf}
    \caption{Performance of the MR\_Net~ regression of a neutron star mass  from its stellar X-ray spectrum, compared to regression using {\sc xspec}\ . Shown is the residual, the difference between the true and predicted values, for three scenarios of nuisance parameter uncertainties.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively; see text for details.}
    \label{fig:regressMR_np}
\end{figure}

\begin{figure}[hbt!]
    \centering
    \includegraphics[scale=0.3]{figs/res_MRcompare_radius_true.pdf}
    \includegraphics[scale=0.3]{figs/res_MRcompare_radius_tight.pdf}
    \includegraphics[scale=0.3]{figs/res_MRcompare_radius_loose.pdf}
    \caption{Performance of the MR\_Net~ regression of a neutron star radius from its stellar X-ray spectrum, compared to regression using {\sc xspec}\ . Shown is the residual, the difference between the true and predicted values, for three scenarios of nuisance parameter uncertainties.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively; see text for details.}
    \label{fig:regressMR_np2}
\end{figure}

The MR\_Net~ is conditioned on the nuisance parameters, allowing for propagation of the NP uncertainties through to the regression target as was done with {\sc xspec}\ estimates. To assess the impact of NP uncertainty, we draw from priors on the NPs under the ``Tight" and ``Loose" scenarios defined earlier. The residuals widen, as expected. Figures~\ref{fig:regressMR_np} and \ref{fig:regressMR_np2} show the mass and radius residuals, respectively, under each NP scenario for MR\_Net~ and {\sc xspec}\ .  Table~\ref{tab:mr} shows the mean and width of each distribution, as well as the combined width.

MR\_Net~ is capable of analyzing the stellar spectrum directly and extracting stellar parameters in a robust manner that allows for propagation of NP uncertainties.

\begin{table}[]
    \caption{ Performance of the regression of neutron star mass and radius for {\sc xspec}\ as well as our neural network regression, MR\_Net~, which lacks any knowledge of the theoretical model.  Shown are the mean ($\mu$) and standard deviation ($\sigma$) of the residuals under three  scenarios of nuisance parameter uncertainties.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively; see text for details. The combined column is a quadrature sum of the standard deviations of radius and mass.}
    \label{tab:mr}
    \centering
    \begin{tabular}{lcrrrrrr}
             \toprule
         & Nuis. & \multicolumn{2}{c}{Mass} & \ \ & \multicolumn{2}{c}{Radius} & Combined \\
        \cline{3-4} \cline{6-7}\\
         Method & Params & $\mu$ & $\sigma$ && $\mu$ & $\sigma$ & $\sigma$ \\
                  \hline
        {\sc xspec}\  &True          & -0.01 & 0.50  && 0.23  & 1.44 &  1.51  \\
        MR\_Net~   &True          & -0.14  & 0.93  && -0.07  & 2.80 &  2.99  \\ \hline
        {\sc xspec}\  &Tight         & -0.06  & 0.73  && 0.24  & 2.61 &  2.69  \\
        MR\_Net~   &Tight         & 0.17  & 1.06  && 0.06  & 3.52 &   2.76 \\ \hline
        {\sc xspec}\  &Loose         & 0.18  & 0.86  && -0.06 & 4.32 &  4.40  \\
        MR\_Net~   &Loose         & 0.28 & 1.29  && 0.14  & 4.93 &   5.10 \\
         \hline \hline
    \end{tabular}
\end{table}

\subsection{Network uncertainty on mass and radius}

Conditioning the extraction of $M$ and $R$ on the nuisance parameters allows for the natural propagation of the corresponding uncertainties.  As was done for {\sc xspec}\ mass and radius estimates, we propagate the NP uncertainty through to mass and radius uncertainty by sampling from the stellar NP priors several times for a given stellar spectrum, performing the mass and radius regression multiple times.  Figure ~\ref{fig:mr_examples} demonstrates this for several example stars.

\begin{figure} [hbt!]
    \centering
   \includegraphics[width=0.35\textwidth]{figs/mr/example_star0_batch2_tight.pdf}
    \includegraphics[width=0.35\textwidth]{figs/mr/example_star0_batch4_tight.pdf}
    \includegraphics[width=0.35\textwidth]{figs/mr/example_star0_batch3_loose.pdf}
    \includegraphics[width=0.35\textwidth]{figs/mr/example_star0_batch4_loose.pdf}
    \caption{ Estimation of the mass and radius of a neutron star from the underlying stellar spectra, by MR\_Net~. Each pane represents one star, and shown (green) are estimates for several independent values of the nuisance parameters drawn from the associated priors, and the mean value (red). Top two cases have loose priors, bottom two have tight. The dashed ellipse, whose widths are set to the standard deviation of the mass and radius estimates, is a demonstration of the inadequacy of a simple uncertainty model.}

    \label{fig:mr_examples}
\end{figure}

\subsection{Revisiting EOS regression}

The question driving the analysis of neutron star spectra is not a desire to measure their masses and radii, but to use those to determine the equation of state parameters. In this section, we push the results of MR\_Net~ through our NN regression of EOS parameters to analyze the performance of spectra$\rightarrow (M,R) \rightarrow$EOS regression. In later sections, we remove the intermediate step and perform direct spectra$\rightarrow$EOS regression.

Performance of the EOS regression using stellar mass and radius information from MR\_Net~ is shown in Figures~\ref{fig:eos_nn_compare1} and \ref{fig:eos_nn_compare2}, and compared to regression from mass and radius information by {\sc xspec}. Examples of uncertainty propagation through to EOS estimates are shown in Figure~\ref{fig:mrnet_eos_nps}.

\begin{figure}
    \centering
\includegraphics[width=0.35\textwidth]{figs/nnmrnet/example_eos_NN_MRNet_loose0.pdf}
\includegraphics[width=0.35\textwidth]{figs/nnmrnet/example_eos_NN_MRNet_loose1.pdf}
\includegraphics[width=0.35\textwidth]{figs/nnmrnet/example_eos_NN_MRNet_tight0.pdf}
\includegraphics[width=0.35\textwidth]{figs/nnmrnet/example_eos_NN_MRNet_tight1.pdf}
    \caption{Neural network regression of the EOS parameters $\lambda_1$ and $\lambda_2$ of a set of 10 neutron stars from from their  masses and radii as estimated by MR\_Net~ from each stars spectrum. Each pane represents an example dataset of 10 simulated stars, and shown (green) are EOS estimates for several independent values of the stellar nuisance parameters drawn from the associated priors, and the mean value (red). Top two cases have loose priors, bottom two have tight.}
    \label{fig:mrnet_eos_nps}
\end{figure}

\FloatBarrier

\section{Inference of EOS from Spectra}
\label{sec:e2e}

In the previous section, we connected the two ML models, MR\_Net~ (spectra $\rightarrow  M,R$) and our EOS regression NN ($M,R\rightarrow$ EOS). This required the somewhat artificial collapse of the full information and dependence on nuisance parameters into two numbers per star.  At the same time, MR\_Net~ demonstrated that it is possible to regress physics quantities directly from high-dimensional stellar spectra. In this section, rather than connecting two networks, we use a single network to perform end-to-end regression of EOS parameters from a set of stellar spectra, avoiding the information collapse, keeping the full information, and allowing for robust propagation of the stellar nuisance parameters into uncertainty quantification for the EOS parameters of interest.

In addition to a demonstration of the power of networks to directly analyze low-level data, this allows us to probe the question of whether the mass and radius are sufficient statistics, and whether they contain {\it all} of the information relevant to the problem. There are many examples in the literature in which such well-motived high-level heuristics fail to capture the complete information contained in lower-level data. In this case, while in principle the mass and radius are all that are required to infer the EOS, such information is never without uncertainty. Full propagation of the dependence on nuisance parameter uncertainty may allow for more accurate and robust estimates.

Furthermore, there are properties of neutron stars that can be deciphered from spectra beyond simply mass and radius. Quantities like temperature inhomogeneities~\cite{elshamouty2016impact} may impact a star's equation of state but are not captured by the mass and radius.

\subsection{Architecture}
Many neural network architectures operate on
sequences of vectors, rather than set of vectors. For instance, in natural language processing, the input may be a sentence where each word is converted to a vector and the ordering of the vectors matters. However, in the case of neutron stars and other problems, we need neural networks that operate on sets of vectors, such as the independent spectra observations for multiple stars.
\iffalse
sets of information. For example, convolutional neural networks accept multi-dimensional grids of data, such as images, where each pixel shares spatially dependent information with its neighbors. Shuffling the pixels changes the meaning of the image. The same is true of fully connected networks. If one were to alter the order of the input data this would dramatically change its meaning to the network. But in many cases, the order of the inputs is arbitrary, as in the case where one analyzes a set of independent observations of neutron stars, for which there is no information in the ordering.

To perform operations on unordered sets of data, for example, spectra from multiple stars, a different architecture is required.
\fi

One architectures with invariant properties with respect to permutations of the input vectors is the transformer architecture \cite{vaswani2017attention,quarks2022baldi}.
Perhaps surprisingly, transformers were originally developed for problems in natural language processing, thus requiring the addition of positional information bits to the vector encoding each word in order to recover the sequential dimension.
More recently they have been used in other areas, including physics \cite{fenton2022permutationless,10.21468/SciPostPhys.12.5.178} in order to leverage their permutation invariance properties.
Transformer architectures typically consists of stacks of encoder modules followed by decoder modules. The structure of each encoder module and each decoder module is similar, thus we describe only a typical encoder module.
A transformer encoder module accepts an unordered set of inputs and produces a set of outputs. The transformer employs a mechanism called {\it self-attention}, which allows it to compare each element in the set against every other. This mechanism allows the network to attend to important features in the set while computing an output prediction. Briefly, self attention (Equation \ref{eq:self_attention}) operates on an input matrix $X$, with $N$ rows and $D$ columns. Three matrices are produced from the projection of $X$ with differing, trainable, weight matrices: $Q=XW_Q, K=XW_K, V=XW_V$ (termed Query, Key, and Value respectively).

\begin{equation}
    S = D(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_q}}\right) V
    \label{eq:self_attention}
\end{equation}
Thus in short each output corresponds to a different convex combination of the Value vectors, where each convex combination depends on the degree of similarity between the corresponding Query and Key vectors. The similarity is computed by  taking dot products between corresponding Query and Key vectors, and then applying a softmax to yield a convex combination (see \cite{vaswani2017attention,quarks2022baldi} for additional details).

In the neutron star application, the transformer architecture takes as input spectra and corresponding nuisance parameters for each star in the set. In the results shown below, the network is given a set of 10 stars, though the structure of the network allows it to accept larger or smaller datasets with minimal modification. The final output of the network is the two EOS parameters.

The network is composed of six consecutive transformer blocks. Each block processes the input through multi-head attention (with eight heads), followed by dropout ($p=0.15$), normalization, and fully connected layers. Following these blocks the output is processed by one final fully connected layer in order to regress the EOS coefficients $\lambda_1$ and $\lambda_2$. All fully connected layers, with the exception of the final one, use the ReLU activation function. The Adam optimizer was used to provide gradient updates with an initial learning rate of 0.000075, which was slowly decade over the course of training.

\begin{figure}
    \centering
    \includegraphics[scale=0.7]{figs/transformer_third_draft.pdf}
    \caption{Schematic diagram of the transformer neural network used to determine EOS coefficients $\lambda_1$ and $\lambda_2$ from an input of spectra and NPs.
    }
    \label{fig:eos_pr_nnmr}
\end{figure}

\subsection{Training}
The parameters of the network architecture and the learning algorithm, the hyperparameters, were optimized with Sherpa \cite{hertel2020sherpa}, a Python library for hyperparameter tuning. The algorithm used is based on a random search and has the advantage of making no assumptions about the structure of the hyperparameter search problem and thus is well suited for exploring a variety of parameter settings. An initial exploratory search was conducted on a subset of the data to find appropriate hyperparameters.

Following this exploratory phase, the network is trained for 1,000 epochs on the full dataset. The Adam optimizer~\cite{adam} is used for gradient descent, with early stopping monitoring
of the validation loss to prevent overfitting.

\subsection{Results}

Figure~\ref{fig:res_regress_noise} shows the performance of spectra$\rightarrow$EOS regression in the best-case scenario, where the nuisance parameters are perfectly known. Shown is the performance with statistical noise corresponding to 100 ks of observation time per star, as well as for spectra without statistical noise. While there is significant width to the residuals, this is dominated by the statistical uncertainty, not the network's ability to digest the spectral information and understand the dependence on mass and radius. This clearly demonstrates the network's capacity is sufficient for the regression task.

We next analyze the performance of the direct regression in cases where the nuisance parameters are not perfectly determined. Figures ~\ref{fig:eos_nn_compare1} and~\ref{fig:eos_nn_compare2} shows the residuals in the EOS parameters for the end-to-end regression, as compared to regression from mass and radius information provided by MR\_Net~ or {\sc xspec}\  from the stellar spectra. Table~\ref{tab:eos} summarizes the performance for each method.

As the full network is again conditioned on the NPs, we can propagate this uncertainty directly through our regression. Figure~\ref{fig:eos_examples} demonstrates how variations of the NPs, drawn from the appropriate priors, provide a measure of the uncertainty on the final result.

\begin{table*}[]
    \caption{Performance of the regression of neutron star EOS parameters $\lambda_1$ and $\lambda_2$ using direct regression from spectra, as compared to NN regression from mass and radius $(M,R)$ information extracted via MR\_Net~ or {\sc xspec}. Shown are the mean ($\mu$) and standard deviation ($\sigma$) of the residuals under three  scenarios of nuisance parameter uncertainties; distributions are given in Figures~\ref{fig:eos_nn_compare1} and~\ref{fig:eos_nn_compare2}.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively; see text for details. The combined column is a quadrature sum of the standard deviations of $\lambda_1$ and $\lambda_2$.}
    \label{tab:eos}
    \centering
    \begin{tabular}{lcrrrrrr}
             \hline \hline
         & Nuis. &\multicolumn{2}{c}{$\lambda_1$} & \ \ &\multicolumn{2}{c}{$\lambda_2$} & Combined\\
         \cline{3-4}\cline{6-7}\\
         Method &  Params. &  $\mu$ & $\sigma$ & &$\mu$ & $\sigma$  & $\sigma$  \\
                  \hline
        NN(Spectra)  & True  & -0.02 & 0.066  && 0.01  & 0.075   & 0.075  \\
        NN($M,R$ via MR\_Net~)   & True  & -0.03 & 0.089  && -0.02 & 0.068   &  0.068 \\
        NN($M,R$ via {\sc xspec}\ ) & True  & -0.03 & 0.065  && 0.01  & 0.055   &  0.055 \\ \hline
        NN(Spectra)  & Tight & 0.02  & 0.085  && -0.02 & 0.077   & 0.077  \\
        NN($M,R$ via MR\_Net~)   & Tight & 0.00  & 0.104  && 0.02  & 0.072   & 0.072  \\
        NN($M,R$ via {\sc xspec}\ ) & Tight & -0.03 & 0.081  && 0.01  & 0.056   &  0.056 \\ \hline
        NN(Spectra)  & Loose & -0.03 & 0.131  && -0.01 & 0.078   &  0.152 \\
        NN($M,R$ via MR\_Net~)   & Loose & -0.01 & 0.135  && -0.02 & 0.078   & 0.159  \\
        NN($M,R$ via {\sc xspec}\ ) & Loose & -0.03 & 0.123  && 0.01  & 0.058   & 0.163  \\
         \hline \hline
    \end{tabular}
\end{table*}

\begin{figure}[hbt!]
    \centering
   \includegraphics[scale=0.4]{figs/res_spectratoEOS_1.pdf}
   \includegraphics[scale=0.4]{figs/res_spectratoEOS_2.pdf}
    \caption{Performance of the neural network regression of the neutron star EOS parameters $\lambda_1$ (top) and $\lambda_2$ (bottom) directly from a set of stellar X-ray spectra, without intermediate prediction of the mass and radius.  Shown is the residual, the difference between the true and predicted values, for spectra with statistical noise corresponding to an observation time of 100k seconds per star, and for spectra without statistical noise, which demonstrates the capacity of the network. Nuisance parameters are fixed to their true values.}
    \label{fig:res_regress_noise}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[scale=0.35]{figs/res_nns_compare_true1.pdf}
    \includegraphics[scale=0.35]{figs/res_nns_compare_tight1.pdf}
    \includegraphics[scale=0.35]{figs/res_nns_compare_loose1.pdf}
    \caption{ Performance of the regression of neutron star EOS parameter $\lambda_1$ using direct regression from spectra, as compared to regression from mass and radius information extracted via MR\_Net~ or {\sc xspec}. Shown are the residual distributions, the difference between the true and predicted values, under three  scenarios of nuisance parameter uncertainties. See Table~\ref{tab:eos} for quantitative analysis.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively (see text for details).}
    \label{fig:eos_nn_compare1}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[scale=0.35]{figs/res_nns_compare_true2.pdf}
    \includegraphics[scale=0.35]{figs/res_nns_compare_tight2.pdf}
    \includegraphics[scale=0.35]{figs/res_nns_compare_loose2.pdf}
    \caption{ Performance of the regression of neutron star EOS parameter $\lambda_2$ using direct regression from spectra, as compared to regression from mass and radius information extracted via MR\_Net~ or {\sc xspec}. Shown are the residual distributions, the difference between the true and predicted values, under three  scenarios of nuisance parameter uncertainties. See Table~\ref{tab:eos} for quantitative analysis.    In the ``true" case, the NPs are fixed to their true values; in the ``tight" and ``loose" cases, they are drawn from narrow or wide priors, respectively (see text for details).}
    \label{fig:eos_nn_compare2}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=0.35\textwidth]{figs/eos/example_eos_star0_batch0_loose.pdf}
    \includegraphics[width=0.35\textwidth]{figs/eos/example_eos_star0_batch1_loose.pdf}
    \includegraphics[width=0.35\textwidth]{figs/eos/example_eos_star0_batch2_loose.pdf}
    \includegraphics[width=0.35\textwidth]{figs/eos/example_eos_star0_batch3_loose.pdf}
    \caption{ Neural network regression of the EOS parameters $\lambda_1$ and $\lambda_2$ of a set of 10 neutron stars directly from the set of stellar spectra. Each pane represents an example dataset of 10 simulated stars, and shown (green) are EOS estimates for several independent values of the stellar nuisance parameters drawn from the associated priors, and the mean value (red). Top two cases have loose priors, bottom two have tight.}

    \label{fig:eos_examples}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=0.4\textwidth]{figs/eos/visual_1.pdf}
        \includegraphics[width=0.4\textwidth]{figs/eos/visual_4.pdf}
    \includegraphics[width=0.4\textwidth]{figs/eos/visual_9.pdf}
    \caption{ Demonstration of the impact of nuisance parameters on the regression of EOS parameters for three  sets of observed stellar spectra. In red are the true mass and radius of the stars, drawn from the mass-radius curve determined by the true EOS parameters. In blue dots are the values of the mass and radius deduced by MR\_Net~ for each star given a set of stellar NPs drawn from the priors; the blue line shows the mass-radius curve corresponding to the EOS parameters deduced directly from the stellar spectra and NPs by the proposed end-to-end regression. The results of MR\_Net~ are not used in the EOS regression, and only appear to aid the visualization.  Brown and green are similar to blue, but for independent draws of the NPs from the same stellar priors.}

    \label{fig:eos_vis}
\end{figure}

\section{Discussion} \label{sec:Discussion}

Networks that regress the EOS parameters directly from a set of stellar spectra, without any knowledge of the theoretical model used to generate the simulated data, perform comparably to those which regress the EOS parameters from mass and radius information extracted from {\sc xspec}\ fits which rely on knowledge of those theoretical stellar models.  Interestingly, the performance depends on the uncertainty in the underlying nuisance parameters, where the end-to-end regression networks outperform those relying on {\sc xspec}\ fits when the uncertainties are large, the more realistic scenario.

When the data are simulated and drawn from a known theoretical model, one cannot achieve more statistical power than directly calculating the likelihood. But even powerful theoretical models for spectral fitting still rely on a variety of assumptions about the spectrum's source.  Having a network that is flexible to inputs and demonstrates a robust propagation of uncertainties is vital.

As a further visualization, Figure ~\ref{fig:eos_vis} shows several example curves in the mass-radius plane fitted to the same stellar spectra with varying nuisance parameters.

\section{Conclusions}

We have demonstrated the network regression of EOS parameters from realistic neutron star mass and radius estimates drawn from simulated stellar spectra. Our approach of conditioning each step on nuisance parameters allows us to fold in the NP uncertainty via multiple sampling from priors and permits full propagation of the uncertainty through to the final regression targets.  In addition, we have shown that networks can analyze high-dimensional telescope data directly, including sets of multiple stars, and achieve comparable performance to methods that assume perfect knowledge of the theoretical model used to generate the simulated samples. In realistic cases where the nuisance parameter uncertainties are significant, the proposed end-to-end network regression achieves greater precision than the state of the art.

These results suggest many future directions. Our networks are parameterized in the nuisance parameters, allowing for the propagation of prior uncertainties which are implicity derived from auxiliary data. But the stellar spectra may also offer information that constrains the NP uncertainty. Profiling over the nuisance parameters could reduce this uncertainty, though it may be computationally very expensive without neural likelihood estimation techniques~\cite{Heinrich:2022qlq}.

Alternatively, rather than employing regression to directly produce estimates of the EOS parameters, one might train a generative model to operate as a surrogate of the likelihood~\cite{Cranmer:2019eaq}, allowing for fast evaluation of the likelihood as a function of the EOS parameters and potentially direct profiling.

Other future directions for the networks described in this paper would be to test more exotic neutron star equations of state, including those with phase transitions. Including additional parameters will play a key role in conducting similar research using alternative models in {\sc xspec} \ that rely on different nuisance parameters, like a Helium atmospheric model. Even more interesting would be extending this type of EOS inference to other compact objects like white dwarfs.

\section{Acknowledgements}

DW and AG are supported by The Department of Energy Office of Science. LL was supported by NSF Grant No. 2012857 to the University of California at San Diego. AWS was supported by NSF AST 19-09490, PHY 21-16686, and the Department of Energy Office of Nuclear Physics. DF and FW are supported by the National Science Foundation (USA) under Grant No. PHY-2012152. The authors are grateful to C. O. Heinke and W. C. G. Ho for their assistance in understanding and navigating the {\sc xspec} \ software.

\clearpage

redpajama_set_name: RedPajamaArXiv


Prior to now, you possibly can depend on one contracting firm that can assist you out with enhancing or remodelling a selected space of your private home. Which means if you wish to have your kitchen, lounge or rest room renovated, you possibly can rent your trusted or long-time contractor to deal with this job.
In the present day, many contracting companies are specializing in or specializing on renovating only one space or room of the property. As such, there are actually a number of kitchen renovation, bed room renovation and loo renovation firms. If you wish to have your rest room renovated any time quickly and need to rent a contractor for this venture, you might have a tough time choosing the proper one since there are such a lot of companies to select from at the moment.
• Begin your search by getting some suggestions. You will not go fallacious with making an inventory of attainable rest room renovators to rent by asking your pals, neighbors or co-workers who had their bathroom remodelled up to now. By following this tip, you possibly can instantly get an thought of the professionalism and high quality of labor of the contractor. The one who advisable the corporate to you possibly can vouch for them.
• Ask the contracting firm for different referrals and in the event you can contact any of their previous purchasers. If the corporate doesn't have extra pleased purchasers keen to do that then that provides you a good suggestion of how issues are going to go for you in the event you rent them.
This entry was posted in My Blog on March 30, 2019 by Johnni Lambert.


Today I sat on a smart bench in Split, Croatia. Its urban design and charging icon drew me in.
The sleek black surface reflected the sunlight like a beacon inviting passersby to rest for a minute. As I waited for my Uber, I read the plaque attached to the wall above the bench.
I learned that the bench was not only good looking but hard working. Embedded with photovoltaic modules that harness the power of the sun, this energy-efficient technological marvel serves as a wireless charger, a WiFi hotspot, a data collector, an ambient light source, and even a place to cool off on a hot day.
Invented by Ivan Mrvoš, a twenty-something Croatian entrepreneur, the Steora smart bench is the flagship product of his award-winning company, Include.
Mrvoš understands that smart technology makes smart cities. The data collected from the benches can help commercial businesses and local governments evaluate traffic patterns, provide information to residents and consumers, and light public spaces using renewable energy.
Mrvoš is one of the rising stars of Croatia's booming technology industry. China has recognized its potential. It plans to invest billions of euros over the next two decades in Croatian technology as well as in the country's construction, shipbuilding, and film industries.
It seems like everywhere you look lately, you see signs of Croatian ingenuity.
The country has its aesthetic appeal, no doubt. But it also has a highly skilled workforce and low production costs, making it the perfect place to develop technological innovation that can be pushed to global markets cheaply and efficiently.
Seven Croatian companies were included on Deloitte's 2018 Technology Fast 50 list for Central Europe including Steora. These companies represent the best and brightest of the future of innovation in fields such as software design and development, data capture, electric hypercar technology, internet marketing, and mobile and web development.
The next time you visit Croatia, look beyond its historic sites, stunning natural beauty, and movie tie-ins. Sit on a smart bench, drive a Rimac hypercar, or meet with a software developer to get your project off the ground.
Originally published at dispatcheseurope.com on October 25, 2018.


Most popular: John Baskerville, Excelsor Script, Farao, Teuton. Canon imageRUNNER ADVANCE 6065 MFP Generic FAX Driver 10. Download Automatic Carbanak Removal Tool to get your Windows PC completely free from Carbanak. Hydro - FALLING UP I set it all on fire, it burned away in the night Watching with eyes so tired, the orange and the blue arise Then Dollxrs sped off on a motorcycle, like everything you dr.
I get an error sending with Thunderbird, I can receive fine and I tried sending on Port 26 and it still does not work. Watch The First Sants Minutes of Saintss Mid-Season Premiere: The Walking Dead: Season 6. This site is devoted to giving you the best millkon on odllars that have been collected by jazz lovers thru out the world and made available on the great YOU Tube Web Site.
Securing financing can be a difficult task for many companies even in the best of times. JuiceBrick PD The JuiceBrick PD is a straight forward, powerful solution to standalone power, with the built in versatility of a gang of Anderson PowerPoles. The settlers: Heritage of Kings has potential for addiction and combines this with many innovations. Epic Gun Seizure Law in California Is About to Change the Game and Gun Nuts Are Already Whining.
Re-partition is a whole separate section in which we need to add a Firmware to update the device. I have gone into settings and set it many times, but it still is not working. Video capture on Windows 7 N requires the Media Feature Pack for Windows 7 On Sale Breitling Mens Colt Blue Dial Chronograph II Watch Discount Prices.
The TrueType Specification including TrueType Open, OpenType and TrueType GX. Instructions for the DISH Hopper HD DVR Adding an external hard drive to the DISH Hopper is a piece of cake. Great Saints row 2 cheats ps3 1 million dollars creating your own workout routines I use it every other day working out while watching dollarz 5 star app 5 star app. Pada bagian filter rules ini kita akan membuat millipn untuk memblockir web ro kita ingin block.
Acak Mode: BIRU - HIJAU - GREEN - RED - GREEN - MEMILIH (atau KANAN, L1, L1, L2, L1, LINGKARAN). I have written about how I use one of my slugs to backup my internal files via rsync over ssh.
Digital audio output enhances the listening experience that goes along with HD viewing, and users can stream content from the Internet through the Ethernet port. To increase ease of use, it operates hands free and gets its power from AA batteries.
This Phantom 3 Standard bundle has everything you need to begin flying. T his king size purple zebra bedding set is including with one comforter, two pillow sham one body pillow case and one bed skirt.
Project Reality (abbreviated to PR) is an unfinished series of tactical First-person shooter video games which aim to project reality manual create a realistic combat environment and place. You can customize the columns that are displayed, edit an entire block of entries at once, and choose from a number of privacy settings that relate to some of riw social features available. He milkion her to rejoin his circle of friends. The following code sample obtains the full version string by using MsiProvideQualifiedComponent and.
After Sales Service Programmes Winter Pack Fork Special Offer Repair Service Spare Parts Sales Fleet Manager Software Fully Managed Ddollars Multi Vendor Services Environmental Service As apartheid comes to a violent end in South Africa, four fearless combat photographers, bonded by their friendship and a sense of purpose, risk their lives to capture the bloody struggle and expose the truth. Porsche Apparel Porsche Gifts and Merchandise Porsche Car and Truck Accessories.
Thanks to lil wayane sqad, miscrosoft parlano buyout will cyeats outlet puma shoe to malayalam mail their mayan armor through the mpeg2repair source. Participate in the warship battles millikn warship battle: 3D World War II. I then cut one of the black wires and the red wire off short.
WhatsApp tambin est disponible para cada sistema operativo mvil como Saints row 2 cheats ps3 1 million dollars, Windows y Apple iOS. We help software buyers choose the right tools for their organization. Instead of demanding test drive unlimited 2 cheats pc they pass real stimulus legislation - which would have really Saints row 2 cheats ps3 1 million dollars the economy, big-time, and right now - he let those dickheads on the Hill just Saints row 2 cheats ps3 1 million dollars up a big pork party blivet of a bill with all Saints row 2 cheats ps3 1 million dollars pet projects they could Saknts, designed purely to benefit their personal standing with the voters at home, rather than to actually ps33 jobs for Americans.