text:
# Find the center, vertices, foci and eccentricity

• May 5th 2010, 09:00 AM
rowdy3
Find the center, vertices, foci and eccentricity
Find the center, vertices, foci and eccentricity of the ellipse given by the following problem.
x^2 / 11 + y^2 / 36 =1. Did I do it right? Thanks
Center: (0,0)
Vertices: (0,6) and (0,-6)
Foci: (0,5) and (0, -5)
eccentricity= 5/6
a^2=36 a=6
b^2=11 b=square root of 11 = 3.32
a^2=b^2+c^2
36=11+c^2
25=C^2
c=5
e=c/a = 5/6
• May 5th 2010, 09:41 AM
bigwave
values of ellipse
$
\frac{\left(x - h\right)^2}{b^2} + \frac{\left(y - k\right)^2}{a^2} = 1
\Rightarrow
\frac{x^2}{11} + \frac{y^2}{36} = 1
$

major axis is parallel to the y axis
Center is $(h,k)\ h=0,\ k=0,\ \Rightarrow(0,0)$
Vertices are $(h,k+a)\ (h,k-a)\ 0+6,\ 0-6 \Rightarrow \pm6$
Foci $(h,k+c)\ (h,k-c) \ 0,0+5\ 0,0-5 \Rightarrow (0,5)\ (0,-5)$
eccentricty $\frac{e}{c}\ \Rightarrow \frac{5}{6}$

format: plain


text:
# 18.9.2 Algorithms (Matrix Smoothing)

Matrix smoothing is done by shrinking and expanding the matrix. The order of shrink/expand operations depends upon matrix dimensions:

• If either the number of columns or rows is less than or equal to 31: The matrix is first expanded so that the row number and the column number are both twice that of the original. Then the expanded matrix is shrank to the original size.
• If both the number of columns and rows is greater than 31: The matrix is first shrank and then expanded to the original size.
• The matrix expansion is implemented by the mexpand X-Function (biquadratic interpolation is used).
• The matrix shrinking is done by the mshrink X-Function. The average value of N adjoining cells (N = Column Factor x Row Factor with default = 4) is calculated to obtain each cell value of the output matrix. Through this process of expanding and shrinking, the size of the output matrix will be exactly the same as the original matrix but the data will be smoothed.


text:
# Sample_ppc shape

Hi.
So I am using an observed data set that N units long. I define my model, define my ObservedRV and give it the training set, and then sample it. Everything appears to work up to that point, the fit parameters of my model look reasonable for the data set I gave it (it is a set we understand well), and then I try to do a posterior predictive sample using it. This is where things start to have problems.

The input set is N units long, (shape (N,))
If I want to get 1 sample from my model, i would expect to use

sample_point = pm.sample(model=model, samples=1, size=1, trace=trace)

but when I do this, I get a strange shape out.

post_pred['output'].shape
(1,N)

Why is it retaining the shape of the training data?

more detailed code example

train.shape
(N,1)
train = train.values.reshape(1,len(train))[0]
train.shape
(756,)
model = pm.Model()
with model:
sigma=pm.HalfNormal('sigma', 0.1)
nu=pm.HalfNormal('nu',1)
mu=pm.Normal('mu',0.0)
r=pm.StudentT('output', mu=mu, nu=nu, sd=sigma, observed=train)
step=pm.NUTS(vars=[r,mu,nu,sigma], gamma=0.25)
trace=pm.sample(draws=2000,step=step,tune=500, chains=20, cores=9)

post_pred = pm.sample_ppc(model=model, samples=1, size=1, trace=trace)
post_pred['output'].shape
(1,N)

My expected result is

post_pred['output'].shape
(1,)

Actually the old behavior outputs (1, ), but I recently changed it.

The main reason for changing it is to have a consistent behavior regarding sampling and prior samples. Before my change, if you want to do prior samples (ie, sample from the prior distribution and prior predictive distribution), you sometimes need to write down the shape specifically:

r = pm.StudentT('output', mu=mu, nu=nu, sd=sigma, observed=train, shape=train.shape)

as the observed variable does not inherent the shape from observed. This is quite inconvenient as user need to change the way model is written when they are sampling from the prior or posterior.

Now, this come to the question of why would we want the observed random variable have the same shape as the input observed. It is a bit of a subtle point, the key is the likelihood function of the parameters. By that I mean the likelihood function change as you have more or less data, thus number of observation (i.e., shape) change the geometry of the likelihood function (you can have a look at my recent talk @ pydataberlin https://github.com/junpenglao/All-that-likelihood-with-PyMC3/tree/pydata_Berlin). On the computation side you want the actual observed and the generated (from prior or posterior) fake observed to have the same shape, then when you are passing the (actual or fake) observed array around and evaluating them on some function (eg the likelihood function), the result is consistent.

I hope this clears things up a bit.

1 Like

This is a little difficult for me to wrap my head around immediately, but I think I get the gist of why this is the default behaviour.

I think I want to ask my question in more detail to see if you can help me understand what a correct model looks like, and a correct prediction.

My intent is to take waterflow data (time series with cyclic patterns), train a model on it, and do predictions/probabilities for the next month. The training set is years of USGS waterflow data.
I want an input of 5,000 measurements over 10 years. I want outputs to be the probabilities for the waterflow for the next month.

This set is one I chose to try and teach myself how to use PyMC3, and to introduce myself to Bayesian modelling. I do have a strong background in MonteCarlo simulations in physics with Geant4, but am trying to branch out into a more general framework.

I have several milestones I have set for myself in this project.

1. Model the flow (Time independent).
2. Produce a probability measurement for the flows for the next 30 days using the time independent model (10,000 simulations).
3. Produce a time dependent model with annual cycles.
4. Produce a probability measurement for the flows for the next 30 days using the time dependent annual cylces model.
5. Produce a time dependent model with 4 seasonal cycles per annual cycle
6. Produce a probability measurement for the flows for the next 30 days using the time dependent annual model with seasonal cycles.

The current status of my project is a very simple model (time independent, StudentT distribution for waterflow levels), and I just wanted to train the model on these values, and predict the next 30 points. That is why I found it very concerning when I could only get 5000 points.

What would the correct way for me to approach milestone 2 on my project plan be? Being able to produce specific prediction sets will be a requirement for me to work on the later stages of this project.

Edit: I do want to apologize if this question is too naive. My introduction to Baysian statistics has been very shallow thus far.

Many way to do so, but the general idea is that you are trying to predict future data (eg, flows of the next 30 days). sample_ppc gives you the “predictive” distribution of the observed, but not the prediction of the future (PS, there are many interesting discussions regarding to the doubt use of data in the Bayesian sense). What you want is p(y_{new}| x, x_{new}, \theta ) with \theta being the posterior of the parameters, in another word, the new predictor (time in this case) need to be incorporate into the prediction.

The easiest way (in PyMC3 sense) is to use a theano.shared variable for the predictor and observed, some example here: http://docs.pymc.io/notebooks/posterior_predictive.html#Prediction
Otherwise, you can generate a new model or add new node in the original model for prediction, example here: https://junpenglao.xyz/Blogs/posts/2017-10-23-OOS_missing.html

What you might also find useful is the Gaussian Process Mauna Loa example: http://docs.pymc.io/notebooks/GP-MaunaLoa.html which is time series model with also prediction on the future.

1 Like

Thank you. The Mauna Loa example looks exceptionally relevant to my current project.

If you would let me ask one more question (to check my understanding),

pm.sample does not predict. It uses the data to determine the best fit parameters of the model.
pm.sample_ppc samples does not predict the future, but rather tries to predict the data that was used to build the model. This tells us weather our model looks like our data? (Is it a good model?).

To predict future data, I need to train a model (for a time dependent series with cyclic behaviours, this will be a multivariate distribution) that is built correctly with sample (I can check this by comparing the observed data with results from sample_ppc), and then I need to separately sample this distribution to create predictions/simulations of possible future behaviour?

Am I understanding the intended workflow of PyMC3 a bit better?

Edit: Also, thank you for posting the videos to your Machine Learning Workshop in May. They are proving most informative.

Yes - but fitting might not be the best description here. Rather think of the sample as a representation of your posterior (i.e., gives you information how the underlying space, or the space that is the most relevant when you trying to take expectation, looks like)

Exactly, and measurement like LOO or WAIC formalized these intuitions.

Yes that’s the conceptual workflow of a Bayesian analysis

1 Like


text:
# Include a CDF file in sites.google

I have a website on sites.google and I also have blog on wordpress. I would like to include some CDF files on them. How can it be done?

-
not on Wordpress.com but possible on other sites running Wordpress. –  cormullion Feb 24 '13 at 20:12
Welcome to Mathematica.SE! I suggest the following: 1) As you receive help, try to give it too, by answering questions in your area of expertise. 2) Read the FAQs! 3) When you see good Q&A, vote them up by clicking the gray triangles, because the credibility of the system is based on the reputation gained by users sharing their knowledge. ALSO, remember to accept the answer, if any, that solves your problem, by clicking the checkmark sign` –  Vitaliy Kaurov Feb 25 '13 at 19:33

This free video course available from Wolfram Training site goes through details of CDF web embedding:

Computable Document Format: Deploying Ideas to the Web

Word Press CDF embedding and some other platforms are discussed in detail. sites.google needs to be checked - but if it does work, then it will use same ideas as explained in the course. Most platforms I know use the same set of procedures. There are two main cases to be aware:

1) Advanced site: your site can host files and allow JS plugins - like full Word Press platform hosted on your own domain. Then you host CDF on your own site and use one of two: - Play directly with JS: Web Delivery: Embedded and Full-Screen CDF - or use Wolfram CDF Plugin form wordpress.org - Here is an example

2) Basic generic blog: your site cannot host files and/or does not allow JS plugins. Then external storage and iframe tags can help. Here is an example

You also maybe interested to take a look at: CDF and MediaWiki. All of this is discussed in details in the linked video course.

-
What you did in the cdfexamples.blogspot.com is precisely the kind of thing I want, can you please elaborate how to do it? –  lovespeed Feb 25 '13 at 20:58
@SthitadhiRoy the 1st link to the video I gave does exactly that. did you try to watch it? –  Vitaliy Kaurov Feb 25 '13 at 21:05


text:
# Cov. right-exact additive functors that don't commute with direct sums?

## Background

Recently, I have been writing up some notes on derived functors and I came across the Eilenberg-Watts theorems [1], which essentially explain why it is hard to find derived functors besides Ext and Tor. Before asking my question, allow me to briefly state these theorems.

Let $R$ and $S$ be rings and $M$ and $S-R$ bimodule. A basic property of the functor $M\otimes_R-:R-\mathrm{Mod}\to S-\mathrm{Mod}$ is that it is an additive covariant right-exact functor. In fact, it also commutes with direct sums. Curiously these properties are enough to characterize it: any covariant additive $T:R-\mathrm{Mod}\to S-\mathrm{Mod}$ that is right-exact and commutes with direct sums is in fact naturally equivalent to some $M\otimes_R -$ for some $S-R$ bimodule $M$. This is the statement of the Eilenberg-Watts theorem for tensor functors.

For completeness, I should state the other version for Hom. If $T:R-\mathrm{Mod}\to S-\mathrm{Mod}$ is an additive left-exact contravariant functor which converts direct sums into direct products (i.e. $T(\oplus M_i) \cong \prod T(M_i)$) then there is an $R-S$ bimodule $M$ such that $T$ is naturally equivalent to $\\mathrm{Hom}_{R}(-,M)$.

Finally, if $T:R-\mathrm{Mod}\to \mathbb{Z}-\mathrm{Mod}$ is left-exact covariant that commutes with inverse limits then there is a left $R$-module $M$ such that $T$ is naturally equivalent to $\mathrm{Hom}_{R}(M,-)$.

## The Prompting for the Question

From the above, any functor satisfying the hypotheses of these Eilenberg-Watts type theorems are going to be naturally equivalent to either a tensor or a Hom, and thus its derived functors will just be Tor or Ext respectively. For instance, if $G$ is a group then $G-\mathrm{Mod}\to \mathbb{Z}-\mathrm{Mod}$ given by $A\mapsto A_G$, where $A_G$ is the quotient of $A$ by the submodule generated by $ga - a$ for all $g\in G$ and $a\in A$ is just the usual coinvariant functor, whose left derived functors are the homology groups $H^i(G,A)$. By Eilenberg-Watts, $-_G$ must be equivalent to some tensor functor, and in fact it is easy to prove that $A\mapsto A_G$ is naturally equivalent to $A\mapsto \mathbb{Z}\otimes_{\mathbb{Z}G}A$

Incidentally, the proof given by C.E. Watts is explicit enough so that the above natural equivalence is apparent.

## The Question

Notice that in each of these the hypothesis of playing nice with limits is required. I am actually interested in functors which do not play nicely with limits. For instance,

What are some examples of covariant right-exact functors $T:R-\mathrm{Mod}\to S-\mathrm{Mod}$ that do not commute with all direct sums? [Edit: $T$ also should not be left exact in this case.]

Such a $T$ of course cannot be a left-adjoint for otherwise it would commute with direct sums. Such a $T$ could be interesting because its left-derived functors may not be "like" the Tor functor. The question also goes for dropping the playing-nice-with-limit hypotheses in the other forms of the theorem. I tried a Google search but could not seem to find anything relevant.

Since I am asking for a list of examples, I have made this a community wiki. Thanks!

[1] Watts, "Intrinsic Characterizations of Some Additive Functors". Proceedings of the American Mathematical Society, Vol. 11, No. 1 (Feb., 1960), pp. 5-8

Thanks everyone for their answers; I think I should have been more precise and asked a question more along the lines of:

What are some derived functors that are not Ext or Tor?

Which I believe some of the existing answers are. In essence I wanted examples that were neither tensors nor Homs in disguise...

-

Here is a specific example, though it admits obvious generalizations. Let $R=S=\mathbb{Z}$, and consider the functor from abelian groups to abelian groups defined by $$T(X) = \mathrm{Ext}^1_{\mathbb{Z}}(\mathbb{Q}_p/\mathbb{Z}_p,X),$$ where $p$ is a prime. This is right-exact since $\mathrm{Ext}^2_{\mathbb{Z}}\equiv0$. It does not commute with direct sums, a fact which is clear when you observe that $T$ is also the $0$th left derived functor of $p$-completion, and $p$-completion does not preserve infinite sums, even when applied to free abelian groups.

-
Nice example. Reading between the lines, by $p$-completion you mean inverse limit of tensor product with $\mathbb Z/p^k$, and by $n$th left derived functor of $F$ you mean $n$th homology of $F$ applied to a projective resolution. But I don't believe I've ever heard that term used this way. I mean, in my experience that construction is only applied when $F$ is right exact (so that the $0$ derived is $F$ again); or if it is applied it doesn't necessarily called derived functors. What also strikes me ... –  Tom Goodwillie Apr 11 '12 at 21:39
... is that the really "good" homotopy-invariant version of $p$-completion is a right derived of a left derived: homotopy inverse limit of derived tensor with $\mathbb Z/p^k$. –  Tom Goodwillie Apr 11 '12 at 21:40

Double dual (when $R=S$ is semisimple).

Or $X\mapsto \Pi_j (M_j\otimes_R X)$ where $\lbrace M_j\rbrace$ is an infinite collection of nontrivial bimodules.

-

What about representable functors? $\hom(M,-) : \mathrm{Mod}(R) \to \mathrm{Ab}$ is right exact iff $M$ is projective, and it preserves direct sums iff $M$ is even finitely generated projective. Of course this is almost a special case of the excellent answer by Tom Goodwillie, but I wanted to add this simple observation.

Remark: If $M$ is not projective, it is harder to characterize those modules $M$ such that $\hom(M,-)$ preserves direct sums, see this MO discussion and the associated nlab article.

-
Thanks Martin. I think I should have been more precise by asking for functors which are neither Hom nor tensor in disguise, so I could get derived functors than are neither Ext nor Tor. However, using this I guess we get examples of exact functors that do not preserve arbitrary sums... –  Jason Polak Apr 11 '12 at 18:53
Sure, I've read your question. But since covariant Hom-functors are always left exact, one computes their right derived functors, namely Ext groups. But Eilenberg Watts (2nd form, which is equivalent the 1st form by GAFT) talks about contravariant Hom-functors. So my example isn't really the Hom-functor in disguise in your question. –  Martin Brandenburg Apr 11 '12 at 19:25

Given associative rings $R$ and $S$, any covariant additive functor $R{-}proj\to S{-}mod$ from the category of projective $R$-modules to the category of $S$-modules can be extended to a right exact functor $R{-}mod\to S{-}mod$ in a unique way. The new functor preserves the infinite direct sums if and only if the original functor did.

So any additive, but not infinite direct sum-preserving functor $R{-}proj\to S{-}mod$ leads to a kind of example you are looking for. One can replace the projective modules with the free ones here if one wishes. Similarly, a covariant additive functor $R{-}inj\to S{-}mod$ from the category of injective $R$-modules to the category of $S$-modules can be extended to a left exact functor $R{-}mod\to S{-}mod$ in a unique way.

The following specific examples come from the theory of comodules and contramodules.

Let $A$ and $B$ be finite-dimensional associative algebras over a field $k$, and let $M$ be an $A$-$B$-bimodule. Then there is a naturally defined covariant right exact functor $F_M\colon A{-}mod\to B{-}mod$ constructed as follows. Given a left $A$-module $P$, consider the two maps $Hom_k(Hom_k(A,M),P)\simeq Hom_k(M,\:A\otimes_kP)\rightrightarrows Hom_k(M,P)$, one of which is induced by the map $M\to Hom_k(A,M)$ defined by the rule $m\mapsto (a\mapsto am)$, while the other one is induced by the action map $A\otimes P\to P$. These are two morphisms of left $B$-modules, the $B$-module structures being induced by the right action of $B$ in $M$. The $B$-module $F_M(P)$ is defined as the cokernel of the difference of of the above two morphisms.

Being defined as the cokernel of a morphism of exact functors, the functor $F_M$ is clearly right exact. One can compute that $F_M(A\otimes V) \simeq Hom_k(M,V)$ for any $k$-vector space $V$, so $F_M$ does not preserve infinite direct sums if $M$ is infinite-dimensional.

The functor $F_M(P)$ is called $Cohom_{A^*}(M,P)$, where $A^*$ and $B^*$ are viewed as (finite-dimensional) $k$-coalgebras, $M$ is an $A^*$-$B^*$-bicomodule, $P$ is considered as an $A^*$-contramodule, and the functor takes values in $B^*$-contramodules. Viewed in this way, the functor is defined for any (not necessarily finite-dimensional) $k$-coalgebras in place of $A^*$ and $B^*$.

Similarly, given a $B$-$A$-bimodule $N$, one defines a covariant left exact functor $G_N\colon A{-}mod\to B{-}mod$ as follows. For any left $A$-module $L$, consider the pair of maps $N\otimes_k L\rightrightarrows N\otimes_k A^*\otimes_k L$ induced by the maps $N\to N\otimes_k A^*$ and $M\to A^*\otimes M$ dual to the action maps $N\otimes_k A\to N$ and $A\otimes_k L\to L$. This is a pair of morphisms of left $B$-modules with the $B$-module structures coming from that of $N$. Set $G_N(L)$ to be the kernel of the difference of this pair of maps.

Being defined the kernel of a morphism of exact functors, the functor $G_N$ is obviously left exact. One can compute that $G_N(Hom_k(A,V)) \simeq N\otimes_k V$, so $G_N$ does not preserve infinite products if $N$ is infinite dimensional. The functor $G_N(L)$ is otherwise known as the cotensor product $N\square_{A^*}L$.

-

A left-exact covariant example is local cohomology: Let $(A, \mathfrak{m})$ be a commutative local ring. The functor $H^0_{\mathfrak{m}}$ from $A$-mod to $A$-mod is $$H^0_{\mathfrak{m}}(M) = \bigcup_{n \geq 1} \{ x \in M : \mathfrak{m}^n x=0 \}$$

This is left exact covariant. For each finite $n$, the corresponding term of the union is $\mathrm{Hom}_A(A/\mathfrak{m}^n, M)$, but there is no $A$-module $N$ such that $H^0_{\mathfrak{m}}(M) = \mathrm{Hom}_A(N,M)$. And, indeed, it does not commute with inverse limits: If $A$ is the $p$-adics, then $H^0_{\mathfrak{m}}(\mathbb{Z}/p^k) = \mathbb{Z}/p^k$ for every $k$, but $H^0_{\mathfrak{m}}(\mathbb{Z}_p)=0$.

-


text:
This topic is 2975 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Hi, I worked through a number of lazy_foo's SDL tutorials and have wrote some code based on them. However, I'm having difficulty with the IMG_Load function that's part of SDL_image. It displays .bmp files without any issues, but when I try to load a .png file, nothing happens. In the following two lines: loadedImage = IMG_Load( tempFileName.c_str() ); if ( loadedImage != NULL ) loadedImage is NULL after the assignment whenever I use a png file. I thought the whole point of IMG_Load was that it could handle multiple image file types. Am I missing something? Is there more code I could paste to shed more light on the issue? Thanks! [Edited by - matrisking on April 27, 2010 9:02:30 AM]

Share on other sites
Did you try following the code through to where it checks if different image types are enabled? I had this problem before... and the documentation doesn't talk much about it, so I'm guessing you are having the same problem I ran into a long time ago.

Open up SDL_image.h. At the top is the following code:

#ifndef _SDL_IMAGE_H#define _SDL_IMAGE_H#define LOAD_BMP

Add in #define LOAD_PNG to make it like this

#ifndef _SDL_IMAGE_H#define _SDL_IMAGE_H#define LOAD_BMP#define LOAD_PNG

Each file type that SDL_Image loads has a different value that needs to be defined. Look through the code to find them.

Share on other sites
I thought that was only the case if you built the library yourself. If not, make sure you have the relevant 2 extra DLLs - libpng and zlib, I believe.

Share on other sites
That did the trick, I forgot all about those.

Thanks!

• 40
• 12
• 10
• 10
• 9
• Forum Statistics

• Total Topics
631371
• Total Posts
2999611
×


text:
liquidity factor?

Here's an attempt at a liquidity alpha factor (via combination several liquidity alpha factors into one).

25
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
# https://arxiv.org/pdf/1412.5072.pdf

from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.factors import CustomFactor, Returns, SimpleMovingAverage
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import Fundamentals
import quantopian.optimize as opt
from sklearn import preprocessing
from scipy.stats.mstats import winsorize
import numpy as np
import pandas as pd

WIN_LIMIT = 0.05

def preprocess(a):

a = a.astype(np.float64)
a[np.isinf(a)] = np.nan
not_nan_ind = np.argwhere(~np.isnan(a))
if not_nan_ind.size > 0:
a_win = winsorize(a[not_nan_ind], limits=[WIN_LIMIT,WIN_LIMIT])
a[not_nan_ind] = a_win
else:
a = winsorize(a, limits=[WIN_LIMIT,WIN_LIMIT])

a = np.nan_to_num(a - np.nanmean(a))

return preprocessing.scale(a)

def normalize(x):

r = x - x.mean()

return r/r.abs().sum()

def make_factors():

class LIX(CustomFactor):
inputs = [USEquityPricing.high, USEquityPricing.low, USEquityPricing.close, USEquityPricing.volume]
window_length = 21
window_safe = True
def compute(self, today, assets, out, high, low, close, volume):
dv = close[-1,:]*np.sum(volume, axis=0)
r = np.amax(high,axis=0) - np.amin(low,axis=0)
out[:] = preprocess(-np.log10(dv/r))

class ILLIQ(CustomFactor):
inputs = [USEquityPricing.close, USEquityPricing.volume, Returns(window_length=21)]
window_length = 21
window_safe = True
def compute(self, today, assets, out, close, volume, ret):
ilq = np.absolute(ret)/(close*volume)
out[:] = preprocess(-np.sum(ilq,axis=0))

class HH(CustomFactor):
inputs = [USEquityPricing.high, USEquityPricing.low, USEquityPricing.volume, Fundamentals.shares_outstanding]
window_length = 21
window_safe = True
def compute(self, today, assets, out, high, low, volume, n_shares):
dv = np.amin(low,axis=0)*np.sum(volume, axis=0)
r = np.amax(high,axis=0) - np.amin(low,axis=0)
out[:] = preprocess(-n_shares[-1]*r/dv)

class share_turnover(CustomFactor):
inputs = [USEquityPricing.volume, Fundamentals.shares_outstanding]
window_length = 21
window_safe = True
def compute(self, today, assets, out, volume, n_shares):
v = np.sum(volume, axis=0)
out[:] = preprocess(-v/n_shares[-1])

class dollar_turnover(CustomFactor):
inputs = [USEquityPricing.close, USEquityPricing.volume, Fundamentals.shares_outstanding]
window_length = 21
window_safe = True
def compute(self, today, assets, out, close, volume, n_shares):
dv = np.sum(close*volume, axis=0)/close[-1,:]
out[:] = preprocess(-dv/n_shares[-1])

factors = [
LIX,
ILLIQ,
HH,
share_turnover,
dollar_turnover,
]

return factors

def factor_pipeline():

factors = make_factors()

pipeline_columns = {}
for k,f in enumerate(factors):

pipe = Pipeline(columns = pipeline_columns,
screen = universe)

return pipe

def initialize(context):

attach_pipeline(factor_pipeline(), 'factor_pipeline')

# Schedule my rebalance function
schedule_function(func=rebalance,
date_rule=date_rules.every_day(),
time_rule=time_rules.market_close(hours=1),
half_days=True)
# record my portfolio variables at the end of day
schedule_function(func=recording_statements,
date_rule=date_rules.every_day(),
time_rule=time_rules.market_close(),
half_days=True)

def recording_statements(context, data):

record(num_positions=len(context.portfolio.positions))
record(leverage=context.account.leverage)

def rebalance(context, data):

alpha = pipeline_output('factor_pipeline').sum(axis=1)
order_optimal_portfolio(opt.TargetWeights(normalize(alpha)), constraints=[])
There was a runtime error.
23 responses

Interesting one, thanks @Grant! Think you might have found a reason for why Q may not want to license individual (weak) factors, but when combined, the factor of related factors becomes more robust and possibly more interesting?

Wouldn't shs_float_cf be a better liquidity gauge than shares_outstanding ?

@Grant,
a very interesting LIX indicator.
Thanks for sharing.
I created the LIX indicator to see how we can use it to determine market regime.

1
Notebook previews are currently unavailable.

And here's the alpha decay and risk exposure analysis just on two factors LIX and ILLIQ.

0
Notebook previews are currently unavailable.

@ Viridian Hawk -

Wouldn't shs_float_cf be a better liquidity gauge than shares_outstanding ?

Not sure. Could you explain your reasoning?

I just spent a little time Googling factor etfs/smart beta/alpha factors, etc. and came across the idea of liquidity. For example:

https://investor.vanguard.com/etf/profile/VFLQ

There, it says:

Fund invests in stocks with relatively lower measures of trading liquidity.
The Liquidity factor is measured by percentage turnover, dollar turnover, and Amihud illiquidity.

It is worth noting that on Q, one could use minute bar data to construct liquidity (illiquidity) factors. There may be liquidity information not revealed in the daily bars.

“Shares outstanding” includes shares "closely held" by insiders and shares otherwise restricted from being traded. "Float" refers to the shares that can actually be traded freely on the public markets, and as such is probably a better fit conceptually for gauging liquidity.

Here's a first-cut at a liquidity factor. Good? Bad? And why?

0
Notebook previews are currently unavailable.

In my opinion...

The Good:

-Consistent specific IR (not too sporadic drops, or negative specific IR)
-Very little Style tilts
-Scoring most stocks in the QTU using TargetWeights
-Low turnover
-Might have value for certain Sectors?

-Relatively low Returns, both specific and common (specific is what matters, and it looks a bit better than common/total)
-Some ‘extreme’ Sector exposures.
-Seems fairly volatile

The Ugly:

-Sector tilts.

Grant ,

Recently, I spent some time working on the illiquidity ratio (ILLIQ), and found that it behaves very much like various volatility factors.

There were a lot of nans so I changed to

out[:] = np.nansum(ilq)

I used DailyReturns instead of Returns and multiplied the value by 1000 to see the numbers.

Amihud (2002) illiquidity ratio, ILLIQ, is one of the most widely used in the industry and is
the daily ratio of absolute stock return to its dollar volume averaged over some period.

0
Notebook previews are currently unavailable.

Than I decided to test ILLIQ as volatility switch on Yulia Malitskaya conventional momentum winners (W_10).
I even left her magic threshold 0.27.
The results are not the best but very similar to backtests with other volatility factors.

0
Notebook previews are currently unavailable.

I see this as a potential test case to see if Q is really interested in funding a lot more itsty-bitsy alphas under the new signal combination paradigm, as they've said they would. It seems that they should publish a list of 50-100 of such little projects, versus trying to find the goose that laid the golden egg. I'd be glad to get in on the action at some low, but consistent monthly payout, versus shooting for the \$50M grand prize.

I used DailyReturns

Thanks for the tip! I think I had the Amihud (2002) illiquidity ratio incorrect.

Now I have:

    class ILLIQ(CustomFactor):
inputs = [USEquityPricing.close, USEquityPricing.volume, DailyReturns(window_length=2)]
window_length = 253
window_safe = True
def compute(self, today, assets, out, close, volume, ret):
ilq = np.absolute(ret)/(close*volume)
out[:] = preprocess(np.nansum(ilq,axis=0))

with

from sklearn import preprocessing
def preprocess(a):
a = a.astype(np.float64)
a[np.isinf(a)] = np.nan
not_nan_ind = np.argwhere(~np.isnan(a))
if not_nan_ind.size > 0:
a_win = winsorize(a[not_nan_ind], limits=[WIN_LIMIT,WIN_LIMIT])
a[not_nan_ind] = a_win
else:
a = winsorize(a, limits=[WIN_LIMIT,WIN_LIMIT])
a = np.nan_to_num(a - np.nanmean(a))
return preprocessing.scale(a)

Grant ,

It is not necessary to specify a window length for DailyReturns

DailyReturns() is the same as  DailyReturns(window_length=2)

I did not use preprocessing just to see how it looks like originally.

One extension here would be to use the Q minute bar data. I have to think there's more than just thin air in liquidity, if Vanguard (and perhaps others) went to the trouble of launching an ETF. There's probably a bunch of academic and industry research on the topic, or they'd be sticking their necks out pretty far.

Grant -- according to Quantopian's TOS as soon as you've posted an algorithm to the forum they are free to use it license-free -- you basically give up any authorship rights to that code. So I hope for your sake you're keeping the really juicy stuff secret.

@ Viridian Hawk -

It's a hobby so there's really nothing to lose.

Anyway, if you have anything technical to add to this thread, on the topic of liquidity, please contribute.

Glancing at https://investor.vanguard.com/etf/profile/portfolio/vflq suggests that this "factor" ETF may have significant exposure to some "common" factors (e.g. size and value). I'm guessing that even though it is called a "factor" ETF, it is more like a "smart beta" ETF in that it has a constraint to attempt to kinda-sorta track the Russell 3000 benchmark versus actually isolating liquidity while controlling for other common factors.

May be of interest:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2291942

We find that the pricing of the Amihud measure is not attributable to
the construction of the return-to-volume ratio that is intended to
capture price impact, but driven by the trading volume component

@ Antony -

Thanks. I gather that you are an academic finance type. Is there any kind of consensus on liquidity as a factor in academic circles? I imagine that at some point, there was a sort of consensus on the original Fama and French factors being "real" (of course, academics make a living out of arguing over minutiae, so "consensus" is probably too strong of a word); perhaps there is similar acknowledgement that liquidity is not just vaporware.

Not an overly academic thing, @Grant.

If you buy an asset knowing that it's going to be difficult to unwind when you most need to, you'll probably want to pay less for it. And that's where the risk premium is earned.

@ Antony -

O.K., and I guess it works the other way too? Stocks that are easy to trade are a bit over-priced due to the fact that folks are willing to pay extra for the assurance that they can cash out whenever they want.

If you buy an asset knowing that it's going to be difficult to unwind when you most need to, you'll probably want to pay less for it. And that's where the risk premium is earned.

Would it be reasonable then to assume then that heavily shorted stocks would have an offsetting effect, since you'll probably want to short the stock at a higher basis for the same reasons and collect risk premium on the short side?

@ Viridian Hawk -

I think you are suggesting that liquidity may be different from a long versus a short perspective, since there may be a difference in closing out the position. In effect, there could be "long liquidity" and "short liquidity".

For example, the QTradableStocksUS() uses the 200-day median daily dollar volume as a proxy for liquidity (see https://www.quantopian.com/help#quantopian_pipeline_experimental_QTradableStocksUS). However, maybe this tilts the QTU toward long liquidity? But really one needs to be able to go long or short with the same ease, I think, for the kind of fund Q is constructing. It also costs more to short, so there's a barrier to shorting relative to going long.

Does Quantopian have so-called short interest data? Along with relative borrowing rates... Seems like a pretty basic set of data...

I think I'll put this aside for awhile. By the way, I sent an e-mail to Q asking if they'd be interested in a liquidity factor--no response. I guess that means "no" but who knows.

1
Backtest from to with initial capital
Total Returns
--
Alpha
--
Beta
--
Sharpe
--
Sortino
--
Max Drawdown
--
Benchmark Returns
--
Volatility
--
 Returns 1 Month 3 Month 6 Month 12 Month
 Alpha 1 Month 3 Month 6 Month 12 Month
 Beta 1 Month 3 Month 6 Month 12 Month
 Sharpe 1 Month 3 Month 6 Month 12 Month
 Sortino 1 Month 3 Month 6 Month 12 Month
 Volatility 1 Month 3 Month 6 Month 12 Month
 Max Drawdown 1 Month 3 Month 6 Month 12 Month
from quantopian.algorithm import attach_pipeline, pipeline_output
from quantopian.pipeline import Pipeline
from quantopian.pipeline.factors import CustomFactor, DailyReturns
from quantopian.pipeline.data.builtin import USEquityPricing
from quantopian.pipeline.data import Fundamentals, factset
import quantopian.optimize as opt
from scipy.stats.mstats import winsorize
from scipy.stats import rankdata
import numpy as np
import pandas as pd
from quantopian.pipeline.classifiers.morningstar import Sector

WIN_LIMIT = 0.0

def preprocess(a):

a = a.astype(np.float64)
a[np.isinf(a)] = np.nan
not_nan_ind = np.argwhere(~np.isnan(a))
if not_nan_ind.size > 0:
a_win = winsorize(a[not_nan_ind], limits=[WIN_LIMIT,WIN_LIMIT])
a[not_nan_ind] = a_win
else:
a = winsorize(a, limits=[WIN_LIMIT,WIN_LIMIT])

a = np.nan_to_num(a - np.nanmean(a))

a = rankdata(a)
a = a/np.amax(a)
a = a - 0.5

return a/np.sum(np.absolute(a))

def normalize(x):

r = x.rank()
r = r/r.max()
r = r - 0.5

return r/r.abs().sum()

def make_factors():

class ILLIQ(CustomFactor):
inputs = [USEquityPricing.close, USEquityPricing.volume, DailyReturns(), Fundamentals.shares_outstanding]
window_length = 253
def compute(self, today, assets, out, close, volume, ret, n_shares):
ilq = np.absolute(ret)/(close*volume)
a = preprocess(np.nanmean(close,axis=0)*np.nanmean(ilq,axis=0))
st = n_shares/volume
b = preprocess(np.nanmean(st,axis=0))
dt = n_shares/(close*volume)
c = preprocess(np.nanmean(close,axis=0)*np.nanmean(dt,axis=0))
out[:] = preprocess(a+b+c)

factors = [
ILLIQ,
]

return factors

def factor_pipeline():

factors = make_factors()

sectors = [101,102,103,104,205,206,207,308,309,310,311]

pipeline_columns = {}
for k,f in enumerate(factors):
for s in sectors:

pipe = Pipeline(columns = pipeline_columns,

return pipe

def initialize(context):

attach_pipeline(factor_pipeline(), 'factor_pipeline')

# Schedule my rebalance function
schedule_function(func=rebalance,
date_rule=date_rules.every_day(),
time_rule=time_rules.market_close(hours=1),
half_days=True)
# record my portfolio variables at the end of day
schedule_function(func=recording_statements,
date_rule=date_rules.every_day(),
time_rule=time_rules.market_close(),
half_days=True)

def recording_statements(context, data):

record(num_positions=len(context.portfolio.positions))
record(leverage=context.account.leverage)

def rebalance(context, data):

alpha = pipeline_output('factor_pipeline').sum(axis=1)

objective = opt.TargetWeights(normalize(alpha))

order_optimal_portfolio(objective=objective,
constraints=[]
)
There was a runtime error.


text:
Browse Questions

# Find the equation of the hyperbola with focus $(2,2) \; e=2$ and directrise $x+y=9$

$\begin{array}{1 1}(A)\;x^2+y^2+4xy-32x-32y+154=0 \\(B)\;x^2-y^2-4xy-32x-32y+154=0 \\(C)\;x^2-y^2-4xy+32x+32y+54=0 \\(D)\;x^2+y^2+4xy+32x+32y+80=0 \end{array}$

Can you answer this question?

Let $P(x,y)$ be any point on the hyperbola


text:
The sum of two numbers is 2 times their difference. The larger number is 6 more than twice the smaller. How do you find the numbers?

It is $\left(a , b\right) = \left(18 , 6\right)$

Explanation:

Let $a$ be the largest number
and $b$ the smallest number.

Hence we have that

$a + b = 2 \left(a - b\right) \implies a + b = 2 a - 2 b \implies a = 3 b$

and

$a = 6 + 2 b \implies 3 b = 6 + 2 b \implies b = 6$

and $a = 18$


text:
# How vulnerable are SIMD operations to side channel attacks?

For a while, I was experimenting with optimization of several crypto implementations with SIMD (Single Instruction Multiple Data ) made available by C compiler extensions. Since, crypto implementations are all about constant time and fast execution, they are optimized to do the same. SIMD has gained massive importance in this area, because we can execute an instruction, on multiple data points simultaneously. This enables vectorization of performance draining parts of code.

I need not emphasize on the performance of SIMD, but I'm worried about its security, despite the performance benefits that it offers.

For example: The ordinary mull instruction may not be constant time on most platforms. Let's imagine that we have a vectorized mull instruction on that platform that operates on two 128 bit registers and outputs another 128 bit register. Let's assume that the lane size is 8 bytes. So, if the multiplication of two corresponding lanes is not constant time, won't that create a net difference in the entire SIMD operation, even though we are operating on multiple data points simultaneously.

Can this lead to software side channel attacks ?

• Can there be software vulnerabilities to a crypto implementation that uses SIMD ?
• Can some physical sensor trace the operations as well as the contents of a SIMD register by monitoring the CPU ?
• the MUL instruction is constant time on every platform that I know, even SIMD. The problem on the x86 would be cache latency. That's the standard side channel attack. – b degnan Jul 12 '20 at 16:58
• @bdegnan Thankyou, for the reply! Motorola PowerPC 7450 (G4e) has/had variable time mull as stated by Bernstein (DJB) in his research paper on Salsa family of stream ciphers cr.yp.to/snuffle/salsafamily-20071225.pdf – Vivekanand V Jul 12 '20 at 17:08
• It's likely already known by other commenters, but floating point multiplication is highly non-constant time (reference). This doesn't have too many applications to crypto (although there are a few that I know of in lattice crypto). – Mark Jul 12 '20 at 17:20
• Some processors have variable time (integer) multipliers, others have constant time. I believe that the general tendency is that higher-end CPUs (with lots of gates) tend to be constant time; lower end CPUs tend to have variable - of course, you need to check with the CPU manual to be certain. – poncho Jul 12 '20 at 18:57
• @VivekanandV If you look at my other posts, you can imagine how an attack can be done with a picoammeter. – b degnan Jul 12 '20 at 21:14


text:
# Graphs of Normal Probability Distributions

## Presentation on theme: "Graphs of Normal Probability Distributions"— Presentation transcript:

Graphs of Normal Probability Distributions
Section 7.1 Graphs of Normal Probability Distributions

Properties of The Normal Distribution
The curve is bell-shaped with the highest point over the mean, μ.

Properties of The Normal Distribution
The curve is symmetrical about a vertical line through μ.

Properties of The Normal Distribution
The curve approaches the horizontal axis but never touches or crosses it.

Properties of The Normal Distribution
The transition points between cupping upward and downward occur above μ + σ and μ – σ .

Properties of The Normal Distribution
The curve is bell-shaped with the highest point over the mean, μ. The curve is symmetrical about a vertical line through μ. The curve approaches the horizontal axis but never touches or crosses it. The transition points between cupping upward and downward occur above μ + σ and μ – σ .

The Empirical Rule Applies to any symmetrical and bell-shaped distribution Approximately 68% of the data values lie within one standard deviation of the mean.

The Empirical Rule Approximately 95% of the data values lie within two standard deviations of the mean.

The Empirical Rule Almost all (approximately 99.7%) of the data values will be within three standard deviations of the mean.

Application of the Empirical Rule
The life of a particular type of light bulb is normally distributed with a mean of 1100 hours and a standard deviation of 100 hours. What is the probability that a light bulb of this type will last between 1000 and 1200 hours? Approximately 68%

Example Male adult heights in North America are approximately normally distributed with a mean of 70 inches and standard deviation of 4 inches. Find the height that is 3 standard deviations above the mean 2 standard deviations below the mean The percentage of men with heights Between 66 inches and 74 inches The percentage of men with heights Between 70 inches and 74 inches The percentage of men with heights above 78 inches. Solution a inches b inches c. 68% d. 34% e % Assignment 13

Normal Probability Distribution (calc)
The Normal Probability Distribution menu for the TI-83+/84+ is found under  DISTR (2nd VARS). NOTE:  A mean of zero and a standard deviation of one are considered to be the default values for a normal distribution on the calculator, if you choose not to set these values. The Normal Distribution functions: #1: normalpdf     pdf = Probability Density Function This function returns the probability of a single value of the random variable x.  Use this to graph a normal curve.  Using this function returns the y-coordinates of the normal curve.           Syntax:   normalpdf (x, mean, standard deviation)

Normal Probability Distribution (calc)
#2: normalcdf cdf = Cumulative Distribution Function This function returns the cumulative probability from zero up to some input value of the random variable x. Technically, it returns the percentage of area under a continuous distribution curve from negative infinity to the x. You can, however, set the lower bound. Syntax: normalcdf (lower bound, upper bound, mean, standard deviation) #3: invNorm( inv = Inverse Normal Probability Distribution Function This function returns the x-value given the probability region to the left of the x-value. (0 < area < 1 must be true.) The inverse normal probability distribution function will find the precise value at a given percent based upon the mean and standard deviation. Syntax: invNorm (probability, mean, standard deviation)

Example Given a normal distribution of values for which the mean is 70 and the standard deviation is 4.5.  Find:     a) the probability that a value is between 65 and 80, inclusive.     b) the probability that a value is greater than or equal to 75.     c) the probability that a value is less than 62.     d) the 90th percentile for this distribution.        (answers will be rounded to the nearest thousandth)   1a:   Find the probability that a value is between 65 and 80, inclusive. (This is accomplished by finding the probability of the cumulative interval from 65 to 80.) Syntax: normalcdf(lower bound, upper bound, mean, standard deviation) Answer:  The probability is %.

Example cont. 1b:  Find the probability that a value is greater than or equal to 75. (The upper boundary in this problem will be positive infinity.  The largest value the calculator can handle is 1 x 1099.  Type 1 EE 99.  Enter the EE by pressing 2nd, comma -- only one E will show on the screen.)  Answer:  The probability is %. 1c:  Find the probability that a value is less than 62. (The lower boundary in this problem will be negative infinity.  The smallest value the calculator can handle is -1 x 1099.  Type -1 EE 99.  Enter the EE by pressing 2nd, comma -- only one E will show on the screen.)  Answer:  The probability is 3.772%.   1d:  Find the 90th percentile for this distribution. (Given a probability region to the left of a value (i.e., a percentile), determine the value using invNorm.) Answer:  The x-value is

Download ppt "Graphs of Normal Probability Distributions"

Similar presentations


text:
OBJECTIVE

We conducted comprehensive assessments of emotional distress to examine relations with diabetes medication adherence over time.

RESEARCH DESIGN AND METHODS

Ethnically and socioeconomically diverse adults treated for type 2 diabetes completed validated self-reports (SRs) for diabetes distress and depression, were administered semistructured depression interviews, and provided blood samples for A1C. Medication adherence among 104 participants was electronically monitored (EM) over the subsequent 3 months; validated SRs of medication adherence were also obtained. Hierarchical linear regression evaluated independent effects of diabetes distress and depression on adherence.

RESULTS

Mean ± SD 3-month medication adherence was 76.1% ± 25.7% for EM and 83.7% ± 21.9% for SR. Higher levels of SR (P < 0.001) and interview-based (P < 0.05) depressive symptom severity (P < 0.05) and diabetes-related distress (P < 0.01) showed a significant bivariate association with EM and SR nonadherence. Regression models showed baseline diabetes distress was a significant independent predictor of EM (β = −0.29; P = 0.001) and SR adherence (β = −0.24; P < 0.02) at follow-up. SR depression was an independent predictor of EM and SR adherence and reduced the effects of diabetes distress to nonsignificance. Subsequent models indicated this effect was driven by somatic rather than cognitive-affective symptoms of depression. Results were consistent but weaker for interview-based depressive symptoms.

CONCLUSIONS

Findings support diabetes-related distress and depression symptom severity as risk factors for type 2 diabetes medication nonadherence. Somatic symptoms captured by depression measures, but not cognitive-affective symptoms, independently predict nonadherence and should be further investigated as a potential link between emotional distress and nonadherence.

The prevalence of depression has been estimated to be up to twofold greater in individuals with type 2 diabetes than in those without diabetes (1), and depression may have significant consequences for diabetes treatment outcomes (24). If these associations reflect a causal influence of depression on diabetes treatment outcomes, that influence may be explained, in part, by treatment nonadherence and suboptimal self-management. Depressive symptoms have been significantly associated with problematic self-management (5), even at low levels of symptom severity that would not warrant a psychiatric diagnosis (6). How to conceptualize these subclinical symptoms of depression has been a matter of debate (79). Questions include whether elevated scores on such measures that do not rise to the level of a diagnosable disorder should be thought of as symptoms of depression at all.

Considerable evidence supports the validity of a construct for diabetes-related emotional distress that is distinct from a depressive mood disorder and specific to the stress of living with a chronic illness and a demanding self-management regimen (9). Diabetes-related distress reflects the emotional and psychological reactions to the burden and stress associated with diabetes and its management (10). Diabetes-related distress has been empirically distinguished from depression and depressive symptoms in relation to self-management and glycemic control (9,1113). The available literature suggests that diabetes distress is more common, more chronic, and more closely associated with diabetes outcomes than major depressive disorder (MDD) (9). Evidence also suggests that diabetes-related distress can often be mischaracterized as MDD in research studies as a result of an over-reliance on self-report (SR) screening measures with high rates of false positives (7,8). Thus, there is a need for continued research that differentiates depression from diabetes-related distress in relation to diabetes self-management.

Studies illustrate that the majority of patients with type 2 diabetes who endorse elevated levels of depressive symptoms on screening measures do not qualify for a diagnosis of a depressive mood disorder (see, e.g., ref. 11). The overlap between somatic symptoms of depression and diabetes-associated illness may partially explain this discrepancy. For example, a study of adults with type 2 diabetes showed that much of the overidentification of depression by a commonly used screening instrument (14) was explained by symptoms involving impairments in sleep, appetite, and energy level (15). Results from a large, population-based study showed that differences in depressive symptoms between adults with and without diabetes were limited to somatic symptoms; no significant differences were found for cognitive-affective symptoms of depression (16). Further, qualitative analysis of structured clinical interviews for depression showed that patients with type 2 diabetes often discuss somatic symptoms of depression as being caused by diabetes and diabetes-related medications (17).

The current study sought to examine, using rigorous assessment methods, associations among diabetes distress, depression, and medication adherence. Based on previous research showing a linear relationship between depressive symptom severity and diabetes treatment nonadherence, even at subclinical levels (5,6) and the lack of a significant relationship between MDD and type 2 diabetes treatment nonadherence (11), our analyses focus on severity ratings for emotional distress, rather than a diagnosis of MDD. We hypothesized independent effects of diabetes-related distress and depression in relation to medication nonadherence and explored the possibility of differential effects for somatic versus cognitive-affective symptoms of depression in relation to diabetes nonadherence. We expected that these relationships would be consistent 1) across SR and interviewer assessments of depressive symptom severity and 2) across subjective and objective measures of diabetes medication adherence.

### Study Sample and Procedures

Participants were recruited from diabetes specialty and primary care clinics affiliated with a large, urban, academic medical center through clinician referral, clinic screening, posted fliers, and mailings as part of a 3-month longitudinal observational study examining treatment adherence and distress. Participants (N = 120) were evaluated for depression, completed SR questionnaires, had blood drawn, received an electronic bottle cap to track their adherence, and were given $50 compensation. Of the 120 participants, 105 completed the electronic medication monitoring portion of the study and returned for a follow-up assessment, for which they were compensated an additional$50. One participant was excluded because their monitored oral medication was discontinued between study visits, leaving 104 participants for these analyses. Entry criteria included age 18 years or older, a diagnosis of type 2 diabetes, an SR ability to read or write in English, and taking oral medication for diabetes or insulin and taking an oral medication for hypertension or hypercholesterolemia. Participants provided informed consent, and the institutional review board at the Albert Einstein College of Medicine approved this study.

### Measures

#### Psychiatric Diagnoses

The Mini International Neuropsychiatric Interview (MINI), a structured and valid diagnostic interview that assesses various psychiatric disorders based on diagnostic criteria, was used to establish baseline current or previous MDD (18). Interviews were administered by doctoral-level clinical psychology students, who were trained and monitored by a licensed clinical psychologist (J.S.G.) through audiotaped supervision.

#### Diabetes-Related Distress

The 17-item Diabetes Distress Scale (DDS) (10) evaluates distress over the past month, using a scale from 1 (no distress) to 6 (serious distress). A mean score <2.0 indicates little to no distress, 2.0 to 2.9 indicates moderate distress, and ≥3 indicates high distress (19). The instrument has four subscales. Here we focus on the five-item emotional burden subscale (DDS-EB) to avoid overlap with reports of adherence to the treatment regimen (i.e., regimen distress items), and to avoid items that are not specifically representative of the experience of emotional distress (e.g., dissatisfaction with providers and support from significant others). The DDS-EB most clearly measures emotional distress related to diabetes (face validity) and has been previously associated with SR medication adherence among adults with type 2 diabetes (20). In this sample, internal reliability was excellent for the DDS total score (α = 0.95) and for the DDS-EB (α = 0.92); scores from these scales were strongly correlated (r = 0.90; P < 0.001).

#### 9-Item Patient Health Questionnaire

The 9-item Patient Health Questionnaire (PHQ-9) is a brief, valid, SR depression screening measure assessing the frequency of depressive symptoms over the past 2 weeks (14). The PHQ-9 assesses four somatic (sleep, fatigue, appetite, and psychomotor retardation) and five cognitive-affective symptoms (lack of interest, depressed mood, negative self-feelings, concentration problems, and suicidal ideation) that are part of the diagnostic criteria for MDD (21). Total scores ≥10 are considered a positive screen for MDD (14). The internal reliability in this sample was adequate for the total (α = 0.88), cognitive-affective (α = 0.83), and somatic (α = 0.75) scales. The two subscales were strongly correlated (r = 0.74; P < 0.001).

#### Montgomery-Åsberg Depression Rating Scale

Doctoral-level clinical psychology students assessed depression symptom severity via a semistructured interview at baseline, with audiorecorded supervision by a licensed clinical psychologist (J.S.G.). The Montgomery-Åsberg Depression Rating Scale (MADRS) probes depressive symptoms experienced during the previous week (22). Additional open-ended follow-up questions elicit information about the severity and frequency of a specific symptom, rated from 0 to 6. Scores between 31 and 34 are considered moderate depression and those ≥35 are considered severe depression (23). The MADRS has fewer somatic symptoms than other measures of depression severity (24). Intraclass correlation between raters has been previously reported as excellent (r = 0.93) (25). The MADRS was also scored as a composite of cognitive-affective symptoms (apparent sadness, reported sadness, inner tension, concentration difficulties, inability to feel, pessimistic thoughts, and suicidal thoughts) and somatic symptoms (reduced sleep, reduced appetite, and lassitude), consistent with previous studies (26,27). Internal reliability for the total scale score in our sample was good (α = 0.85), with lower reliability for the somatic (α = 0.60) than the cognitive-affective (α= 0.82) subscales, which were strongly correlated (r = 0.66; P < 0.001).

Each participant was given a Medication Event Monitoring System bottle cap (MEMS) (AARDEX Group, Zurich, Switzerland) to track one diabetes-related medication for approximately 3 months. Oral antihyperglycemic medications were tracked when available. If a participant was taking insulin and no oral antihyperglycemic medication, a medication for cholesterol or blood pressure was tracked. If multiple medications satisfied these criteria, the oral medication that was taken most frequently or was the most difficult to remember was selected. The percentage of doses taken as prescribed was calculated by dividing the number of times the bottle was opened by the number of openings prescribed during that time period. MEMS data were corrected only when participants could identify a specific day or days on the calendar that they took their medication in a way that would not have been recorded by the MEMS. MEMS readings were corrected for 44 participants for common reasons including medication taken from a pillbox, inpatient hospitalization, or taking multiple doses out of the bottle when leaving home.

SR adherence ratings were adapted from Lu et al. (28) and were previously validated in adults with type 2 diabetes (29). Participants were asked six separate questions: “What percentage of the time did you take all your diabetes medications as your doctor prescribed?,” using past week, month, and 3-month intervals with 11 response categories (0%, 10%, 20%, . . . 100%), and “On average, how would you rate your ability to take all your diabetes medications as your doctor prescribed?,” using past week, month, and 3-month intervals with 6 response categories (very poor, poor, fair, good, very good, and excellent). For the current analyses, responses were combined into a composite standardized score (α = 0.97).

#### Glycemic Control

Diabetes control was assessed by glycated hemoglobin A1c (HbA1c) via a central laboratory. Participants had blood drawn by a research nurse at each study visit. Only 101 participants supplied a valid HbA1c result at the follow-up for reasons including refusal because of a recent blood draw as part of regular clinical care, a research nurse was unavailable during the scheduled visit, and unwillingness to wait or return for a blood draw.

### Data Analysis

Descriptive statistics, including distribution normality, were examined for demographic and study-related variables. Bivariate relations among covariates, indicators of emotional distress, SR/electronic monitoring (EM) adherence, and HbA1c values were examined using the Pearson r statistic. Hierarchical linear regression examined independent effects of baseline indictors of emotional distress in predicting 3-month SR/EM adherence. The DDS-EB, total continuous scores from the PHQ-9 and MADRS, and composite scores of cognitive-affective and somatic depressive symptoms were evaluated as independent predictors of EM and SR adherence in separate models. Multicollinearity in the multiple regression models was assessed using the variance inflation factor and tolerance, and no significant multicollinearity among predictors was found. All analyses were completed using SPSS software version 22.0 (IBM Corp., Armonk, NY).

### Sample Characteristics

Baseline characteristics of the larger sample were described by Baek et al. (30), and values for baseline variables included in this analysis were not significantly different between those who are included in the current analysis and those who were lost to follow-up. As seen in Table 1, participants were, on average, ethnically diverse, older adults; 51.9% reported at least one diabetes-related complication and 38.5% were prescribed insulin. The average EM percentage of dosage taken as prescribed was 76.1% (SD 25.7%); the average number of days monitored was just over 3 months (96.1 ± 13.4 days). The average DDS score indicated “moderate” diabetes distress. Approximately 46.2% of the sample scored above the threshold for clinically significant diabetes distress (19). Approximately 21% screened positive for MDD on the PHQ-9. However, only five individuals who screened positive on the PHQ-9 met diagnostic criteria for current MDD (22.7%); three positive screens met criteria for previous MDD (13.6%). The small number of participants with current MDD precluded further analysis of the role of diagnosis in this sample.

Table 1

Participant characteristics (N = 104)

 Age (years) Mean (SD) 56.6 (9.2) Range 29–87 Sex Female 64.4 (67) Male 35.6 (37) BMI, mean (SD) 35.7 (7.8) Hispanic ethnicity (n = 93) 26.9 (25) Education level (n = 103) Less than high school diploma 17.5 (18) High school diploma 16.5 (17) Some college 34.0 (35) College degree 19.4 (20) Some graduate school or degree 12.6 (13) Yearly family income (n = 98) <$10,000 18.4 (18)$10,000–14,999 14.3 (14) $15,000–24,999 19.4 (19)$25,000–49,999 28.6 (28) $50,000–99,999 18.4 (18)$100,000–149,999 1.0 (1) Years since diagnosis (n = 102), mean (SD) 12.5 (9.3) Baseline A1C, mean (SD) Percentage 7.8 (1.6) Millimoles per mole 61.2 (17.1) Prescribed insulin 38.5 (40) MDD diagnosis based on the MINI (n = 103) Current 7.7 (8) Past 12.5 (13) Total diabetes distress, mean (SD) 2.3 (1.2) Clinically significant diabetes distress 46.2 (48) Emotional burden 2.5 (1.5) Total PHQ-9 score, mean (SD) 5.9 (5.6) Screened positive 21.2 (22) Somatic symptoms 3.4 (2.9) Cognitive-affective symptoms 2.5 (3.1) Total MADRS score, mean (SD) 10.1 (9.0) Somatic symptoms 3.5 (3.5) Cognitive-affective symptoms 6.6 (6.3) Self-reported medication adherence Percentage-based ratings, mean (SD) 83.7 (21.9) Categorical ratings Very poor 0 (0) Poor 6.7 (7) Fair 8.7 (9) Good 19.2 (20) Very good 27.9 (29) Excellent 37.5 (39)
 Age (years) Mean (SD) 56.6 (9.2) Range 29–87 Sex Female 64.4 (67) Male 35.6 (37) BMI, mean (SD) 35.7 (7.8) Hispanic ethnicity (n = 93) 26.9 (25) Education level (n = 103) Less than high school diploma 17.5 (18) High school diploma 16.5 (17) Some college 34.0 (35) College degree 19.4 (20) Some graduate school or degree 12.6 (13) Yearly family income (n = 98) <$10,000 18.4 (18)$10,000–14,999 14.3 (14) $15,000–24,999 19.4 (19)$25,000–49,999 28.6 (28) $50,000–99,999 18.4 (18)$100,000–149,999 1.0 (1) Years since diagnosis (n = 102), mean (SD) 12.5 (9.3) Baseline A1C, mean (SD) Percentage 7.8 (1.6) Millimoles per mole 61.2 (17.1) Prescribed insulin 38.5 (40) MDD diagnosis based on the MINI (n = 103) Current 7.7 (8) Past 12.5 (13) Total diabetes distress, mean (SD) 2.3 (1.2) Clinically significant diabetes distress 46.2 (48) Emotional burden 2.5 (1.5) Total PHQ-9 score, mean (SD) 5.9 (5.6) Screened positive 21.2 (22) Somatic symptoms 3.4 (2.9) Cognitive-affective symptoms 2.5 (3.1) Total MADRS score, mean (SD) 10.1 (9.0) Somatic symptoms 3.5 (3.5) Cognitive-affective symptoms 6.6 (6.3) Self-reported medication adherence Percentage-based ratings, mean (SD) 83.7 (21.9) Categorical ratings Very poor 0 (0) Poor 6.7 (7) Fair 8.7 (9) Good 19.2 (20) Very good 27.9 (29) Excellent 37.5 (39)

Data are presented as percentage (n) unless otherwise indicated.

### Bivariate Relationships: Covariates, Medication Adherence, and Emotional Indicators

Among potential covariates, two were associated with greater EM and SR adherence: older age (r = 0.33, P = 0.001 and r = 0.21, P = 0.03, respectively) and fewer days of electronic monitoring (r = −0.36, P < 0.001 and r = −0.13, P = 0.20, respectively). Participants taking insulin had significantly lower EM adherence (67.24% ± 27.28%; P = 0.005), compared with those taking oral medications only (81.72% ± 23.09%), although their SR adherence was not significantly different (P = 0.59). Significant covariates were included in the respective multivariable models presented below.

EM and SR adherence at follow-up were significantly correlated (r = 0.45; P < 0.001); each was significantly associated with baseline HbA1c (r = −0.24, P = 0.02 and r = −0.32, P < 0.001, respectively) and with follow-up HbA1c (n = 101; r = −0.30, P = 0.003 and r = −0.31, P = 0.001, respectively). DDS-EB was positively correlated with PHQ-9 (r = 0.62; P < 0.001) and MADRS total scores (r = 0.45, P < 0.001). DDS-EB was somewhat more closely related to cognitive-affective (PHQ-9: r = 0.65, P < 0.001; MADRS: r = 0.46, P < 0.001) than somatic depressive symptoms (PHQ-9: r = 0.50, P < 0.001; MADRS: r = 0.32, P = 0.001). MADRS and PHQ-9 total scores were strongly correlated (r = 0.82; P < 0.001).

Those who screened positive on the PHQ-9 had significantly lower EM adherence (62.52% ± 29.21%; t = 2.91; P = 0.005) than those who screened negative (79.80% ± 23.49%). This difference was significant for SR adherence as well (P = 0.010). Each of the continuous variables for emotional distress was significantly and negatively associated with EM adherence in bivariate analyses, except for MADRS cognitive-affective symptoms (P = 0.16); all emotional distress variables were significantly associated with SR adherence (Table 2). No significant bivariate relationships were found between emotional distress variables and HbA1c at baseline (data not shown). Only DDS total scores were significantly associated with follow-up HbA1c (n = 101; r = 0.22; P = 0.029).

Table 2

Correlations for indicators of emotional distress and medication adherence

DDS
Total −0.37 −0.28
DDS-EB −0.38 −0.28
PHQ-9
Total −0.39 −0.40
Somatic symptom scale −0.45 −0.38
C/A scale −0.27 −0.37
Total −0.22* −0.26
Somatic symptom scale −0.32 −0.20*
C/A scale −0.14NS −0.25
DDS
Total −0.37 −0.28
DDS-EB −0.38 −0.28
PHQ-9
Total −0.39 −0.40
Somatic symptom scale −0.45 −0.38
C/A scale −0.27 −0.37
Total −0.22* −0.26
Somatic symptom scale −0.32 −0.20*
C/A scale −0.14NS −0.25

C/A, cognitive/affective symptom.

*P < 0.05;

P < 0.01;

P < 0.001; NS, not significant.

### Multivariable Regression Analyses

Hierarchical models showed that greater levels of diabetes distress were significantly associated with lower EM and SR adherence after covariate adjustment (Table 3, step 1). When the PHQ-9 total score was added to these models, it accounted for significant additional variance in both EM (P = 0.021) and SR adherence (P = 0.004), and attenuated the effect of diabetes distress to nonsignificance (Table 3, step 2a). Adding MADRS total scores in separate models did not account for significant additional variance in EM or SR adherence beyond that accounted for by diabetes distress and covariates (Table 3, step 2b).

Table 3

Diabetes distress and depression symptoms as predictors of medication adherence

EM
SR
βP valueR2βP valueR2
Step 1   0.32   0.10
Age 0.18 0.04  0.14 0.15
Insulin −0.15 0.09  – –
Days monitored −0.31 <0.001  – –
DDS-EB −0.29 0.001  −0.24 0.02
Step 2a   0.36   0.17
Age 0.15 0.08  0.10 0.31
Insulin −0.11 0.20  – –
Days monitored −0.35 <0.001  – –
DDS-EB −0.15 0.14  −0.04 0.75
PHQ-9 total −0.25 0.02  −0.35 0.004
Step 2b   0.33   0.12
Age 0.17 0.05  0.13 0.20
Insulin −0.14 0.10  – –
Days monitored −0.33 <0.001  – –
DDS-EB −0.25 0.01  −0.18 0.10
MADRS total −0.11 0.27  −0.15 0.16
EM
SR
βP valueR2βP valueR2
Step 1   0.32   0.10
Age 0.18 0.04  0.14 0.15
Insulin −0.15 0.09  – –
Days monitored −0.31 <0.001  – –
DDS-EB −0.29 0.001  −0.24 0.02
Step 2a   0.36   0.17
Age 0.15 0.08  0.10 0.31
Insulin −0.11 0.20  – –
Days monitored −0.35 <0.001  – –
DDS-EB −0.15 0.14  −0.04 0.75
PHQ-9 total −0.25 0.02  −0.35 0.004
Step 2b   0.33   0.12
Age 0.17 0.05  0.13 0.20
Insulin −0.14 0.10  – –
Days monitored −0.33 <0.001  – –
DDS-EB −0.25 0.01  −0.18 0.10
MADRS total −0.11 0.27  −0.15 0.16

P values <0.05 and β values are set in boldface to indicate significance.

Next, we examined somatic and cognitive-affective depressive symptom dimensions as predictors of adherence, controlling for covariates. Results showed that only somatic depressive symptoms were independently associated with EM adherence, whereas cognitive-affective symptoms were not independently significant in either model. Neither of the depression symptom components was independently predictive of SR adherence; this was consistent for MADRS and PHQ-9 (Table 4, steps 1a and 1b).

Table 4

Somatic and cognitive-affective depressive symptom dimensions as independent predictors of medication adherence

EM
SR
βP valueR2βP valueR2
Step 1a   0.40   0.17
Age 0.19 0.02  0.10 0.28
Insulin −0.08 0.38  — —
Days monitored −0.34 <0.001  — —
PHQ-9 S −0.50 <0.001  −0.23 0.10
PHQ-9 C/A 0.13 0.29  −0.17 0.22
Step 1b   0.31   0.09
Age 0.21 0.02  0.16 0.10
Insulin −0.14 0.11  — —
Days monitored −0.33 <0.001  — —
MADRS S −0.28 0.02  −0.04 0.77
MADRS C/A 0.03 0.78  −0.20 0.13
Step 2a   0.40   0.17
Age 0.15 0.07  0.11 0.26
Insulin −0.07 0.38  — —
Days monitored −0.35 <0.001  — —
DDS-EB −0.15 0.11  −0.10 0.36
PHQ-9 S −0.34 <0.001  −0.30 0.006
Step 2b   0.36   0.11
Age 0.16 0.07  0.13 0.20
Insulin −0.11 0.19  — —
Days monitored −0.33 <0.001  — —
DDS-EB −0.24 0.01  −0.21 0.04
MADRS S −0.20 0.03  −0.11 0.29
EM
SR
βP valueR2βP valueR2
Step 1a   0.40   0.17
Age 0.19 0.02  0.10 0.28
Insulin −0.08 0.38  — —
Days monitored −0.34 <0.001  — —
PHQ-9 S −0.50 <0.001  −0.23 0.10
PHQ-9 C/A 0.13 0.29  −0.17 0.22
Step 1b   0.31   0.09
Age 0.21 0.02  0.16 0.10
Insulin −0.14 0.11  — —
Days monitored −0.33 <0.001  — —
MADRS S −0.28 0.02  −0.04 0.77
MADRS C/A 0.03 0.78  −0.20 0.13
Step 2a   0.40   0.17
Age 0.15 0.07  0.11 0.26
Insulin −0.07 0.38  — —
Days monitored −0.35 <0.001  — —
DDS-EB −0.15 0.11  −0.10 0.36
PHQ-9 S −0.34 <0.001  −0.30 0.006
Step 2b   0.36   0.11
Age 0.16 0.07  0.13 0.20
Insulin −0.11 0.19  — —
Days monitored −0.33 <0.001  — —
DDS-EB −0.24 0.01  −0.21 0.04
MADRS S −0.20 0.03  −0.11 0.29

P values <0.05 and β values are set in boldface to indicate significance. C/A, cognitive-affective symptom scale; PHQ-9 C/A, Patient Health Questionnaire, 9-item, cognitive-affective symptom scale; PHQ-9 S, Patient Health Questionnaire, 9-item, somatic symptom scale. S, somatic symptom scale.

Based on these results, we tested a final set of models adding somatic symptoms to covariates and diabetes distress to examine their independence in predicting adherence. PHQ-9 somatic symptoms accounted for additional independent variance in EM and SR nonadherence; diabetes distress was attenuated to nonsignificance in each of these models (Table 4, step 2a). MADRS somatic depressive symptoms accounted for additional independent variance in EM adherence, but not in SR adherence; in each case diabetes distress remained a significant independent predictor of adherence (Table 4, step 2b).

Our results show that various indicators of depression and diabetes-related emotional distress were bivariately associated over time with both SR and EM medication nonadherence in a diverse sample of adults with type 2 diabetes. Diabetes distress and depression shared between 20% and 38% of their variance in this sample, suggesting substantial overlap in the measurement of these emotional distress constructs. Cognitive-affective symptoms of depression assessed by clinical interview, the gold-standard approach of depression assessment, were the only indicator of emotional distress not significantly associated with EM adherence in bivariate analysis. About 21% of our sample screened positive for MDD, and their adherence was significantly lower than those who did not screen positive. However, only 36% of these positive screens met diagnostic criteria for either current or previous MDD (64% were false positives). Further, total diabetes-related distress was the only indicator of emotional distress that was associated with HbA1c values at follow-up.

Multivariable adjustment showed that diabetes distress, measured by the emotional burden subscale of the DDS, was significantly associated with both EM and SR adherence. Self-reported depression symptom severity was independently significant across adherence measures and attenuated the contribution of diabetes distress to nonsignificance, indicating overlapping variance between these two measures. Effects were weaker for the MADRS, which was not independently predictive of either EM or SR adherence, and did not attenuate the effects of diabetes distress. Importantly, the MADRS interview was designed to be less sensitive to somatic symptom confounding when chronic illness is present (23). Thus somatic symptoms may play a role in explaining differences between MADRS and PHQ-9 findings.

As recommended by authoritative diagnostic systems (21), MADRS interviews involved clinical judgment in the evaluation of symptoms. Also, the MADRS does not assess increases in sleep or appetite, only decreases. The PHQ-9 includes items for fatigue and psychomotor retardation, whereas the MADRS only assesses difficulty in initiating activities (lassitude). Thus the PHQ-9 more fully assesses the symptom criteria for MDD (21) but may also be more vulnerable to confounding by physical illness and illness-related functional impairment (24). PHQ-9 reports of disturbed appetite, fatigue, and psychomotor retardation were the only symptoms that differentiated adults with diabetes from those without diabetes in a recent population-based study (16). Furthermore, problems with appetite, sleep, and fatigue were the only PHQ-9 symptoms that did not discriminate between false-positive and true-positive cases of depressive disorder in a study of adults with type 2 diabetes (15). This suggests these somatic symptoms are less closely associated with true depression in diabetes than cognitive-affective symptoms. These differences warrant further investigation.

Our findings demonstrating an independent role for somatic symptoms of depression in predicting medication adherence in type 2 diabetes are, to our knowledge, novel and require replication. Previous research has repeatedly linked depression to diabetes treatment nonadherence, but these studies have not examined differences between depression symptom dimensions (5). Furthermore, prior studies have predominantly relied on SR screening measures of depression, for which more than half of positive screens would be false positives (11,31), consistent with our findings. When studies assessed depressive disorders using a standardized diagnostic interview, they generally failed to find a relationship with diabetes treatment adherence (see, e.g., ref. 11). Intervention trials that have been successful in treating depression in diabetes have been unsuccessful in improving medication adherence or self-management (see, e.g., ref. 32). Other data demonstrate that relations between depression symptom severity and problems with self-management, including medication nonadherence, persist after the exclusion of nearly all likely cases of MDD (6). The lack of an independent role for cognitive-affective symptoms of depression reported by this study casts further doubt on whether clinical depression is the construct underlying the relationship between elevations on depressive symptom scales and diabetes treatment nonadherence.

Studies that have attempted to identify independent relationships between diabetes distress and depressive symptoms in relation to diabetes self-care have found somewhat mixed results, emphasizing one or the other construct, with variation of effects across aspects of self-management (12,13,20,33). Our study is unique in 1) using both a clinical interview for depression severity and validated SR; 2) using EM to assess missed medication doses and validated SR; and 3) examining somatic and cognitive-affective components of depression. These methods allow us to limit the role of shared measurement error in these effects. We also took a conservative approach of using only the emotional burden subscale of the DDS in our analyses to limit potential confounding with treatment adherence behaviors and to avoid introducing sources of distress, such as dissatisfying relationships with providers, that could influence adherence through pathways other than emotional distress (10,20).

Our design also has limitations. There is no “gold standard” for the assessment of medication adherence and, although our data are consistent with prior research showing that EM captures more nonadherence than SR, EM is also subject to measurement error; both measures were equivalently related to HbA1c in this and previous research (29). Further limitations of EM include a lack of standardization concerning which type of medication was monitored. Thus, we cannot speak to variations across these measures with certainty, despite our attempt to avoid errors related to EM by adjusting values based on compelling participant reports about doses taken in ways that would not be captured by EM. The same is true for our measures of depression: we cannot be certain about which differences between the assessments explain the variations in effects. The MADRS somatic symptom scale also had marginal reliability. We focused on depressive symptom severity rather than diagnosis of a mood disorder in these analyses based on prior evidence (6,11), and results cannot speak directly to the effect of depressive mood disorders. Our relatively small sample may also limit the generalizability of findings and may have limited power for our analyses, especially for more distal HbA1c effects. Given the correlational nature of this study, it is possible that a third variable, such as disease burden (e.g., complications and comorbidities) or complexity of diabetes management, contributed to both increased emotional distress and nonadherence. Finally, although we captured average adherence over a clinically meaningful period of time (i.e., ∼3 months), our design also prevents an examination of how these variables change over time. Future longitudinal studies with more than two time points and those that use ecological momentary assessment methods may better evaluate the direction of and capture the dynamic relations among the factors identified here.

Distinguishing between somatic and cognitive-affective aspects of depression in future research may improve our understanding of the nature and potential explanatory mechanisms for relations between depression and diabetes health outcomes. Studies from the cardiovascular disease (CVD) literature, which also reports consistent evidence linking depression to poor prognosis, suggest that this distinction could be important. For example, population-level data showed that somatic symptoms of depression, but not cognitive-affective symptoms, were independently associated with coronary heart disease and personal and family history of myocardial infarction, among other CVD risk factors (34). Meta-analysis of prospective studies of depressive symptom dimensions in individuals with CVD found that somatic symptoms were significantly associated with mortality and cardiovascular events in fully adjusted analyses; cognitive-affective symptoms were not (35). Other recent studies similarly demonstrate an independent effect of somatic symptoms, rather than cognitive-affective symptoms, in predicting poor CVD outcomes (see, e.g., refs. 35, 36, and 37). Whether this pattern of findings reflects a unique effect of certain depressive symptoms, a particular depression “subtype,” or confounding with symptoms of physical illness is unclear. Further investigation of these possibilities is needed in relation to diabetes self-management and health outcomes.

### Clinical Implications

One fundamental clinical implication of this study is that providers should be aware that emotional distress can often be associated with risk for type 2 diabetes treatment nonadherence. Our results provide evidence for considerable consistency of effects across measures of emotional distress; they also demonstrate substantial intercorrelation among these indicators of emotional distress. This likely reflects a significant role for shared variance across these measures. While routine screening for depression, an increasingly common standard of diabetes care, identifies patients with more depressive symptoms and who may be at higher risk for treatment nonadherence, it does not separate the larger number of false-positive from true-positive depression cases, nor does it identify the most appropriate intervention. Further evaluation is necessary for screening programs to be effective (7,8). Rather than focusing exclusively on clinical depression—an important but far less prevalent problem than clinically significant diabetes distress in this and other samples (see, e.g., refs. 38 and 39)—evidence suggests that emotional distress that does not rise to the level of a psychiatric diagnosis could nevertheless represent meaningful risk for problems with diabetes treatment adherence. As such, providers should be attuned to emotional distress in their patients during clinical encounters and consider specific depressive symptom presentations and diabetes-related distress as part of their further evaluation of positive screens to more accurately identify the nature of the problem and guide the selection of appropriate interventions (7,9,11).

Consistent evidence across methods of assessment demonstrates that somatic symptoms involving fatigue, sleep, appetite disturbance, and psychomotor changes may warrant particular attention in relation to emotional distress and risk for type 2 diabetes treatment nonadherence. Whether these symptoms represent somatic symptoms of a major depressive episode, residual symptoms between episodes of depression, situational stress, confounding with symptoms of physical illness, or side effects of prescribed medication, or whether they have other explanations can likely only be ascertained through a careful clinical interview. Distinguishing among these explanations for reported “symptoms” should be at least as important as establishing their severity in guiding the selection of appropriate interventions to reduce emotional distress and improve treatment adherence among adults living with type 2 diabetes.

See accompanying articles, pp. 2122, 2126, 2141, 2149, 2158, 2165, 2174, 2190, and 2197.

Acknowledgments. The authors thank the participants of the study for their contributions. The authors also thank the excellent former graduate students who contributed, particularly Drs. Havah Schneider, Abigail Batchelder, Sabrina Esbitt, Rachel Baek, Erica Shreck, Molly Tanenbaum, and Seth Margolis (all were affiliated with Ferkauf Graduate School of Psychology, Yeshiva University, Bronx, NY, at the time of the study).

Funding. This work was partially supported by a pilot and feasibility grant from the Albert Einstein Diabetes Research Center, which was supported at the time of data collection by National Institutes of Health (NIH) grant P60 DK020541. J.S.G. is additionally supported by grants R18 DK09872, R01 DK104845, and P30 DK11022 from the National Institute of Diabetes and Digestive and Kidney Diseases. C.J.H. is supported by The Drs. David and Jane Willner Bloomgarden Family Fellowship Fund.

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. J.S.G. conceived the study design and contributed to every aspect of this article. N.S.K. wrote the initial draft of the manuscript and researched the data. D.H.B. wrote the introduction and discussion sections and edited the manuscript. A.S. wrote the introduction and discussion sections. C.J.H. researched the data and reviewed and edited the final manuscript. J.S.G. is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Prior Presentation. Parts of this study were presented at the 75th Scientific Sessions of the American Diabetes Association, Boston, MA, 5–9 June 2015.

1.
Ali
S
,
Stone
MA
,
Peters
JL
,
Davies
MJ
,
Khunti
K
.
The prevalence of co-morbid depression in adults with type 2 diabetes: a systematic review and meta-analysis
.
Diabet Med
2006
;
23
:
1165
1173
[PubMed]
2.
Lustman
PJ
,
Anderson
RJ
,
Freedland
KE
,
de Groot
M
,
Carney
RM
,
Clouse
RE
.
Depression and poor glycemic control: a meta-analytic review of the literature
.
Diabetes Care
2000
;
23
:
934
942
[PubMed]
3.
de Groot
M
,
Anderson
R
,
Freedland
KE
,
Clouse
RE
,
Lustman
PJ
.
Association of depression and diabetes complications: a meta-analysis
.
Psychosom Med
2001
;
63
:
619
630
[PubMed]
4.
Hofmann
M
,
Köhler
B
,
Leichsenring
F
,
Kruse
J
.
Depression as a risk factor for mortality in individuals with diabetes: a meta-analysis of prospective studies
.
PLoS One
2013
;
8
:
e79809
[PubMed]
5.
Gonzalez
JS
,
Peyrot
M
,
McCarl
LA
, et al
.
Depression and diabetes treatment nonadherence: a meta-analysis
.
Diabetes Care
2008
;
31
:
2398
2403
[PubMed]
6.
Gonzalez
JS
,
Safren
SA
,
Cagliero
E
, et al
.
Depression, self-care, and medication adherence in type 2 diabetes: relationships across the full range of symptom severity
.
Diabetes Care
2007
;
30
:
2222
2227
[PubMed]
7.
Gonzalez
JS
,
Fisher
L
,
Polonsky
WH
.
Depression in diabetes: have we been missing something important?
Diabetes Care
2011
;
34
:
236
239
[PubMed]
8.
Fisher
L
,
Gonzalez
JS
,
Polonsky
WH
.
The confusing tale of depression and distress in patients with diabetes: a call for greater clarity and precision
.
Diabet Med
2014
;
31
:
764
772
[PubMed]
9.
Snoek
FJ
,
Bremmer
MA
,
Hermanns
N
.
Constructs of depression and distress in diabetes: time for an appraisal
.
Lancet Diabetes Endocrinol
2015
;
3
:
450
460
[PubMed]
10.
Polonsky
WH
,
Fisher
L
,
Earles
J
, et al
.
Assessing psychosocial distress in diabetes: development of the diabetes distress scale
.
Diabetes Care
2005
;
28
:
626
631
[PubMed]
11.
Fisher
L
,
Skaff
MM
,
Mullan
JT
, et al
.
Clinical depression versus distress among patients with type 2 diabetes: not just a question of semantics
.
Diabetes Care
2007
;
30
:
542
548
[PubMed]
12.
Gonzalez
JS
,
Delahanty
LM
,
Safren
SA
,
Meigs
JB
,
Grant
RW
.
Differentiating symptoms of depression from diabetes-specific distress: relationships with self-care in type 2 diabetes
.
Diabetologia
2008
;
51
:
1822
1825
[PubMed]
13.
Aikens
JE
.
Prospective associations between emotional distress and poor outcomes in type 2 diabetes
.
Diabetes Care
2012
;
35
:
2472
2478
[PubMed]
14.
Kroenke
K
,
Spitzer
RL
,
Williams
JBW
.
The PHQ-9: validity of a brief depression severity measure
.
J Gen Intern Med
2001
;
16
:
606
613
[PubMed]
15.
Twist
K
,
Stahl
D
,
Amiel
SA
,
Thomas
S
,
Winkley
K
,
Ismail
K
.
Comparison of depressive symptoms in type 2 diabetes using a two-stage survey design
.
Psychosom Med
2013
;
75
:
791
797
[PubMed]
16.
Wiltink
J
,
Michal
M
,
Wild
PS
, et al
.
Associations between depression and diabetes in the community: do symptom dimensions matter? Results from the Gutenberg Health Study
.
PLoS One
2014
;
9
:
e105499
[PubMed]
17.
Tanenbaum
ML
,
Ritholz
MD
,
Binko
DH
,
Baek
RN
,
Shreck
MS
,
Gonzalez
JS
.
Probing for depression and finding diabetes: a mixed-methods analysis of depression interviews with adults treated for type 2 diabetes
.
J Affect Disord
2013
;
150
:
533
539
18.
Sheehan
DV
,
Lecrubier
Y
,
Sheehan
KH
, et al
.
The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10
.
J Clin Psychiatry
1998
;
59
(
Suppl. 20
):
22
33; quiz 34–57
[PubMed]
19.
Fisher
L
,
Hessler
DM
,
Polonsky
WH
,
Mullan
J
.
When is diabetes distress clinically meaningful?: establishing cut points for the Diabetes Distress Scale
.
Diabetes Care
2012
;
35
:
259
264
[PubMed]
20.
Gonzalez
J
,
Shreck
E
,
Psaros
C
,
Safren
S
.
Distress and type 2 diabetes treatment adherence: a mediating role for perceived control
.
Health Psychol
2015
;
34
:
505
513
[PubMed]
21.
American Psychiatric Association
.
Diagnostic and Statistical Manual of Mental Disorders
. 5th ed.
Arlington, VA
,
American Psychiatric Association Publishing
,
2013
.
22.
Montgomery
SA
,
Asberg
M
.
A new depression scale designed to be sensitive to change
.
Br J Psychiatry
1979
;
134
:
382
389
[PubMed]
23.
Müller
MJ
,
Himmerich
H
,
Kienzle
B
,
Szegedi
A
.
Differentiating moderate and severe depression using the Montgomery-Asberg depression rating scale (MADRS)
.
J Affect Disord
2003
;
77
:
255
260
[PubMed]
24.
Svanborg
P
,
Åsberg
M
.
A comparison between the Beck Depression Inventory (BDI) and the self-rating version of the Montgomery Asberg Depression Rating Scale (MADRS)
.
J Affect Disord
2001
;
64
:
203
216
[PubMed]
25.
Williams
JBW
,
Kobak
KA
.
Development and reliability of a structured interview guide for the Montgomery Asberg Depression Rating Scale (SIGMA)
.
Br J Psychiatry
2008
;
192
:
52
58
26.
McIntyre
RS
,
Konarski
JZ
,
Mancini
DA
, et al
.
Improving outcomes in depression: a focus on somatic symptoms
.
J Psychosom Res
2006
;
60
:
279
282
[PubMed]
27.
Reijnders
JSAM
,
Lousberg
R
,
Leentjens
AFG
.
Assessment of depression in Parkinson’s disease: the contribution of somatic symptoms to the clinimetric performance of the Hamilton and Montgomery-Asberg rating scales
.
J Psychosom Res
2010
;
68
:
561
565
[PubMed]
28.
Lu
M
,
Safren
SA
,
Skolnik
PR
, et al
.
.
AIDS Behav
2008
;
12
:
86
94
[PubMed]
29.
Gonzalez
JS
,
Schneider
HE
,
Wexler
DJ
, et al
.
.
Diabetes Care
2013
;
36
:
831
837
[PubMed]
30.
Baek
RN
,
Tanenbaum
ML
,
Gonzalez
JS
.
Diabetes burden and diabetes distress: the buffering effect of social support
.
Ann Behav Med
2014
;
48
:
145
155
[PubMed]
31.
Roy
T
,
Lloyd
CE
,
Pouwer
F
,
Holt
RIG
,
Sartorius
N
.
Screening tools used for measuring depression among people with type 1 and type 2 diabetes: a systematic review
.
Diabet Med
2012
;
29
:
164
175
[PubMed]
32.
Lin
EHB
,
Katon
W
,
Rutter
C
, et al
.
Effects of enhanced depression treatment on diabetes self-care
.
Ann Fam Med
2006
;
4
:
46
53
[PubMed]
33.
Fisher
L
,
Glasgow
RE
,
Strycker
LA
.
The relationship between diabetes distress and clinical depression with glycemic control among patients with type 2 diabetes
.
Diabetes Care
2010
;
33
:
1034
1036
[PubMed]
34.
Michal
M
,
Wiltink
J
,
Kirschner
Y
, et al
.
Differential associations of depressive symptom dimensions with cardio-vascular disease in the community: results from the Gutenberg health study
.
PLoS One
2013
;
8
:
e72014
[PubMed]
35.
de Miranda Azevedo
R
,
Roest
AM
,
Hoen
PW
,
de Jonge
P
.
Cognitive/affective and somatic/affective symptoms of depression in patients with heart disease and their association with cardiovascular prognosis: a meta-analysis
.
Psychol Med
2014
;
44
:
2689
2703
[PubMed]
36.
Hwang
B
,
Moser
DK
,
Pelter
MM
,
Nesbitt
TS
,
Dracup
K
.
Changes in depressive symptoms and mortality in patients with heart failure: effects of cognitive-affective and somatic symptoms
.
Psychosom Med
2015
;
77
:
798
807
[PubMed]
37.
Roest
AM
,
Wardenaar
KJ
,
de Jonge
P
.
Symptom and course heterogeneity of depressive symptoms and prognosis following myocardial infarction
.
Health Psychol
2016
;
35
:
413
422
[PubMed]
38.
Fisher
L
,
Skaff
MM
,
Mullan
JT
,
Arean
P
,
Glasgow
R
,
Masharani
U
.
A longitudinal study of affective and anxiety disorders, depressive affect and diabetes distress in adults with type 2 diabetes
.
Diabet Med
2008
;
25
:
1096
1101
[PubMed]
39.
Nicolucci
A
,
Kovacs Burns
K
,
Holt
RIG
, et al.;
DAWN2 Study Group
.
Diabetes Attitudes, Wishes and Needs second study (DAWN2™): cross-national benchmarking of diabetes-related psychosocial outcomes for people with diabetes
.
Diabet Med
2013
;
30
:
767
777
[PubMed]