How are molecular diagnostic tests evaluated?
Molecular diagnostic testing and pharmacogenomic testing differ in their objectives and possible testing outcomes. Diagnostic testing detects whether an individual is affected with a specific disease or not, and the testing outcome is generally one of two outcomes: positive or negative. In diagnostic testing, sensitivity is the ability of a test to correctly identify a patient who has a disease as having the disease (true positive). Specificity is the ability of a test to correctly identify a healthy individual as not having the disease (true negative). Both sensitivity and specificity have only two possible outcomes: for sensitivity, consider that a diagnostic test may either result in a true positive or a false negative; for specificity, a diagnostic test may either be a false positive or a true negative.
Why is it necessary to evaluate pharmacogenomic tests differently than diagnostic tests?
Psychiatric pharmacogenomics does not have an individual genetic marker or causative gene like is often typical in molecular diagnostic testing; rather, medication response is multifactorial, relying on the interaction of many genes, as well as environmental factors. For this reason, the GeneSight test’s proprietary CPGx® technology takes into account all relevant genes known to be associated with medication response. Consequently, the potential measurements of sensitivity and specificity shift from the association of a genetic marker with response to the association of the GeneSight algorithm with response. Instead of diagnosing the bimodal presence or absence of a disease state, the GeneSight test predicts patient response to medications using a three category system (green, yellow, red), which correlates with the predicted amount of gene-drug interaction for the patient (none, moderate, and significant, respectively). Unlike a two-category system, a three-category system does not lend itself to sensitivity and specificity measurements.
Additionally, a complexity arises in the fact that many patients taking psychiatric medications are not taking just one. Polypharmacy further complicates the multifactorial nature of medication response, since medications may compete for the same target or induce/inhibit relevant metabolizing enzymes. In the context of the GeneSight test, this means patients may be taking medications from each of their green, yellow, and red categories. Consequently, calculating the sensitivity and specificity for each individual medication loses real world application. In the clinical studies that validate the GeneSight test, patients are entered into analyses based on their total medication regiment. This is accomplished by putting a patient in their “worst” medication category. For example, a patient taking one green and one red category medication is entered into the analysis as a red category patient. This approach empowers a more comprehensive analysis of a patient’s overall medication response, rather than analyzing a patient’s response to each medication individually, since it is nearly impossible to determine which medication correlates to specific aspects of a patient’s response.
How are pharmacogenomic tests evaluated?
Pharmacogenomic tests are evaluated by analytical validity (accuracy), clinical validity, clinical utility and economic utility.
|Definition: Analytical validity is defined by the Federal Drug Administration (FDA) as being comprised of the following components: analytical sensitivity, clinical sensitivity, precision, and accuracy.1 When healthcare providers ask about the analytical validity of a genetic test, the component in which they are often most interested is accuracy. Accuracy is the ability of the test to provide the correct results.||Benefit of the GeneSight® Test: The GeneSight Psychotropic test’s accuracy is 99.6% and above for all individual assays and for the testing system as a whole.2 The accuracy of the GeneSight Psychotropic test is regulated by CAP, CLIA, and New York State Department of Health, and its analytical validity has been approved by each of these organizations.|
|Definition: Clinical validity evaluates how well a test predicts patient outcomes. This information is obtained by evaluating patients who received treatment as usual (i.e. their treatment was not guided by pharmacogenomic testing), then applying a pharmacogenomic test retrospectively to assess its ability to identify which patients are least likely to respond.||Benefit of the GeneSight® Test: Four studies have demonstrated the clinical validity of the GeneSight Psychotropic test.3-6 Three of these studies (the prospective clinical outcomes studies) were entered into a pooled analysis that showed patients in the treatment as usual arm who were prescribed red category medications had less improvement in depressive symptoms compared to patients prescribed green or yellow category medications.7|
|Definition: Clinical utility evaluates whether a test influences and benefits patient care. A pharmacogenomic test must be able to demonstrate improved clinical response above treatment as usual to have clinical utility.||Benefit of the GeneSight® Test: Four studies have demonstrated the clinical utility of the GeneSight Psychotropic test.3-5,8 The largest of these, a study with 165 patients undergoing treatment for depression, found that patients whose treatment was guided by the GeneSight Psychotropic test showed a 70% greater improvement in depressive symptoms than people who received treatment as usual.5 Additionally, the GeneSight Psychotropic test was shown to reduce uncertainty and increase confidence in decision-making for pharmacotherapy.9|
|Definition: Economic utility evaluates the overall cost benefit to the healthcare system of implementing a technology.||Benefit of the GeneSight® Test: The GeneSight Psychotropic test has been shown to have economic utility in two studies.6,8 A retrospective study on the economic utility of the GeneSight Psychotropic test demonstrated that, compared to patients who unknowingly taking medications in their green or yellow categories, patients unknowingly taking medications in their red category had significantly more healthcare visits, medical absence days, and disability claims, and had an average annual increase in healthcare utilization cost of $5,188.6 A prospective study found that patients spent an additional $1,035 per year on medications when their treatment was not guided by the GeneSight Psychotropic test.8|
- U.S. Department of Health and Human Services. (2014). FDA Framework for Regulatory Oversight of Laboratory Developed Tests (LDTs): Draft Guidance. Rockville, MD: Food and Drug Administration.
- Luminex Molecular Diagnostics, Inc. (2013). Luminex xTAG® CYP2D6 Kit v3: For In Vitro Diagnostic Use System Manual. Toronto, ON: Luminex Molecular Diagnostics, Inc.
- Winner JG, Carhart JM, Altar CA, Allen JD, Dechairo BM. A prospective, randomized double-blind study assessing the clinical impact of integrated pharmacogenomic testing for major depressive disorder. Discovery Med. 2013;16(89): 219-227.
- Hall-Flavin DK, Winner, JG, Allen JD, Jordan JJ, Nesheim RS, Snyder KA, Drews MS, Eisterhold LL, Biernacka JM, Mrazek DA. Using a pharmacogenomic algorithm to guide the treatment of depression. Transl Psychiatry. 2012;2:e 172.
- Hall-Flavin DK, Winner JG, Allen JD, Carhart JM, Proctor B, Snyder KA, Drews MS, Eisterhold LL, Geske J, Mrazek DA. Utility of integrated pharmacogenomic testing to support the treatment of major depressive disorder in a psychiatric outpatient setting. Pharmacogenetics and Genomics. 2013;23(10):535-548.
- Winner JG, Allen JD, Altar CA, Spahic-Mihajlovic A. Psychiatric pharmacogenomics predicts health resource utilization of outpatients with anxiety and depression. Transl Psychiatry. 2013;3:e300. doi:10. 1038/tp.2013.2.
- Altar CA, Carhart JM, Allen JD, Hall-flavin DK, Dechairo BM, Winner JG. Clinical validity: Combinatorial pharmacogenomics predicts antidepressant responses and healthcare utilizations better than single gene phenotypes. Pharmacogenomics J. 2015;15(5):443-51.
- Winner JG, Carhart JM, Altar CA, et al. Combinatorial pharmacogenomic guidance for psychiatric medications reduces overall pharmacy costs in a 1 year prospective evaluation. Curr Med Res Opin. 2015;31(9):1633-43.
- Altar C, A, Carhart J, Allen J, D, Hall-Flavin D, Winner J, Dechairo B, Clinical Utility of Combinatorial Pharmacogenomics-Guided Antidepressant Therapy: Evidence from Three Clinical Studies. Mol Neuropsychiatry 2015;1:145-155.