Improving Vocational Outcomes of Veterans With Psychiatric Disorders: Career Counseling & Development
Improving Vocational Outcomes of Veterans With Psychiatric Disorders: Career Counseling & Development
The Veterans Health Administration (VHA) offers robust vocational programming that have
 helped countless Veterans obtain competitive employment; however, these services are not
 uniformly effective as recent data suggests that only 35 to 43% of Veterans are competitively
 employed at time of discharge. For those who become competitively employed, job tenure may be
 brief, which is often attenuated by underemployment or poor person-job fit. Moreover, only
 3.5% of Veterans experiencing vocational problems engage in vocational services offered by
 the VHA. On average, it takes Veterans more than four years to utilize vocational services.
 These Veterans are at high risk of acquiring multiple functional losses and developing
 chronic disabilities as their vocational needs go unmet for years.

 Research suggests that intrinsic factors like lacking clear vocational goals, perceiving
 barriers to employment, and negative beliefs about one's ability to work contribute to low
 engagement, outcomes, and tenure of some consumers of vocational rehabilitation services.
 Thus, the VA may be able to improve vocational engagement, outcomes, and tenure of Veterans
 with psychiatric disorders by enhancing vocational services with added interventions
 targeting unhelpful psychological factors. Career counseling and development services have
 been shown to be effective in helping civilian populations clarify vocational goals and
 identity, enhance vocational self-efficacy, and increase proactive vocational behaviors in
 the face of obstacles. Additionally, career counseling and development services help
 facilitate greater "match" between a person and their job, and person-job match is a key
 determinant of long-term career tenure among individual with psychiatric disorders.

 The researchers of this project propose a three-aim study to develop a career counseling and
 development intervention for Veterans with psychiatric disorders (CCD-V). The first aim will
 focus on the design and development of the CCD-V intervention with veteran and provider input
 (n=16). The second aim will pilot test the intervention in an open trial (n=10) to gather
 Veteran input on the initial intervention. The third and final aim will consist of a
 feasibility pilot randomized controlled trial (n=50) to examine acceptability and feasibility
 outcomes and to explore the impact of the CCD-V intervention in terms of functional
 improvement and other vocational outcomes. The proposed CCD-V intervention will consist of
 approximately eight individual sessions that will be offered concurrently with existing VHA
 vocational rehabilitation services, (e.g., transitional work experience [TWE]).The final
 product of this study is to produce a manualized CCD-V intervention, and corresponding
 fidelity monitoring checklist, to be tested later in a larger efficacy trial.
AIM 1: Develop a career counseling and development treatment manual for Veterans with
 psychiatric disorders (CCD-V) along with a fidelity monitoring checklist based on veteran and
 provider needs assessment (n=16).

 Procedures. Manual development will be an iterative process that spans over Aim 1 and Aim 2
 (Open Trial) and will include: 1) reviewing clinical procedures and interventions of the
 Vocational Evaluation Center (VEC) for career counseling and development in vocational
 rehabilitation, related treatment manuals (Opening Doors), and relevant career development
 theory; 2) conducting veteran and provider focus groups to gather perspectives on necessary
 treatment components; 3) developing a preliminary version of the manual through a two-day
 retreat attended by key CDA-2 mentors and consultants, and 4) revising the manual based on
 mentor, provider, and veteran feedback. A fidelity monitoring checklist will also be
 developed to monitor treatment adherence of clinicians implementing the CC&D-V protocol. The
 fidelity monitoring checklist will be used in Aim 2 and Aim 3 of this study to monitor
 treatment adherence of the study clinicians.

 Needs Assessment Focus Groups. Two focus groups will be conducted with veterans (n=8), and
 two focus groups will be conducted with mental health providers (n=8) with knowledge in
 vocational rehabilitation practice (psychologists, social workers, and vocational
 rehabilitation specialists). The providers will be recruited from outside the Bedford VA to
 capture perspectives from clinicians who are not familiar with the Bedford VA Vocational
 Evaluation Center. Half of veteran participants will be recruited from currently enrolled TWE
 veterans. The other half of veteran participants will be individuals who previously
 participated in TWE and were unsuccessful in obtaining employment. Focus groups will be 90
 minutes, and will be audio recorded for future transcription and analysis with NVIVO.
 Transcription will be completed by a VA approved contractor.

 A qualitative interview guide drawing upon Psychology of Working theory, protocol developed
 through the Vocational Evaluation Center (VEC) as well as the Opening Doors manual when
 conducting provider and veteran focus groups. Overall, the goal is to gain an unfiltered
 perspective on career development and factors related to successful vocational recovery,
 while also querying on aspects of existing practices An open-ended interview guide consisting
 of open-ended questions will focus on 1) barriers, potential solutions, and perceived needs
 to promote successful career outcomes among veterans with psychiatric disorders; 2) obtain
 feedback on the useful elements of Psychology of Working theory, as well as the VEC and
 Opening Doors protocols as it relates to their clinical experience/expertise; 3) explore
 issues related to augmenting VA vocational services with career counseling and development
 interventions.

 The purpose of the veteran focus groups are to gather personal accounts from people with
 lived experience on career development/vocational recovery, and to connect that to the best
 available research and clinical practice to inform treatment development. The qualitative
 interview guide will consist of open-ended questions to gather the following information: 1)
 barriers, potential solutions, and perceived needs to promote successful career outcomes
 among veterans with psychiatric disorders; 2) obtain feedback on the useful elements of
 Psychology of Working theory, as well as the VEC and Opening Doors protocols as it relates to
 their lived experiences; 3) gather information regarding preferred methods of implementation
 of adjunctive career counseling and development services.

 Sample. The eight provider participants will be recruited through emails to vocational
 providers and will be scheduled for one of two one hour focus group of providers only during
 regular work hours. These focus groups will be held via teleconference. Half of the eight
 veteran participants will be recruited from the Compensated Work Therapy TWE program, while
 half will be former TWE recipients who did not secure competitive employment. Veteran
 participants will be scheduled for a one of two one hour focus group of veterans only. All
 participants will be consented for participating in the interview and having their
 participation audio-recorded.

 Qualitative Analysis. A Hamilton Rapid Turn-around Analytic Approach will be used. This
 technique is suited to short-term projects using targeted interview guides. It improves on
 past rapid procedures through its systematic, rigorous approach, yielding results that are
 slightly less nuanced but more efficient. The PI will develop an analytic template organized
 by the topic areas from the semi-structured interview guiding qualitative interviews. The PI
 and two other researchers will apply the template to summarize two interviews to test the
 usability and relevance of the template to the transcripts. All three researchers will meet
 to establish consistency in how the template is used to summarize the transcripts. From here,
 all focus groups will be individually summarized (each focus group will be analyzed by two
 researchers to ensure reliability). PI will then transfer all the summary points into a
 master matrix (respondent x topic area) as an important step in drawing conclusions and
 deriving meaning from the transcripts. PI will then condense the summaries in the master
 matrix by identifying and organizing recurring themes. The other two researchers will review
 the condensed matrix to the original master matrix to confirm all central themes have been
 identified and to ensure consensus.

 Two-Day Retreat. Data gathered from the veteran and provider focus groups will be used to
 inform the design of the treatment protocol as well as the fidelity monitoring checklist. The
 retreat will be coordinated at the Bedford VA, and will be facilitated by Drs. Stevenson and
 Mueller and attended by mentors (Drs. Bakken, Blustein, Ellison, and Kelly), as well as
 consultants (Drs. Millner, Kenny, Duffy, Allan, and Autin). Drs. Stevenson and Mueller will
 facilitate the retreat beginning with an opening agenda and ground rules for facilitating
 dialogue and resolving conflicting viewpoints about focus group material, expert knowledge,
 and research evidence. Every theme will be presented and reviewed. Drs. Stevenson and Mueller
 will facilitate conversation about each theme and whether or not thematic content could be
 integrated into the treatment protocol considering the integrity of best available research
 and practice. In situations where consensus among content experts cannot be reached, the
 majority stance will determine inclusion or exclusion of content into the treatment manual.

 Content and Structure of the CCD-V Intervention. Foremost, the intervention will largely be
 based upon the results of the needs assessment focus groups and qualitative analysis
 discussed above. Some of the proposed content is based on Psychology of Working Theory (PWT)
 and the corresponding Psychology of Working Counseling (PWC) model which identifies critical
 areas of intervention in vocationally-oriented treatment (e.g., assessment of work-related
 needs in terms of survival/power, social connection, and self-determination; increasing
 skills related to proactivity and adaptability to navigate career development processes;
 increasing awareness to environmental or other contextual factors influencing career choice
 and development; increasing sense of choice and agency over one's career development; and
 improving relational support systems to enhance vocational career development outcomes). In
 addition, the preliminary treatment goals and interventions for the CCD-V intervention take
 into account current VEC practices among veterans with psychiatric disabilities (e.g.,
 exploring the impact of psychiatric and physical disability on one's work like and career
 choices) as well as the Opening Doors treatment manual (e.g., a person-environment fit model
 of career development that espouses a three step process of knowing oneself, knowing one's
 work-related options, and embarking on a career path that fits with self-knowledge).
 Anticipated length of treatment is eight weekly sessions which is informed by the average
 length of treatment veterans receive through the VEC. Sessions will be delivered to veterans
 through individual, face-to-face weekly meetings with a masters-level (or higher) trained
 counselor.

 Qualitative Inquiry of Initial Treatment Protocol. After developing an initial version of the
 CCD-V manual, qualitative interviews will be conducted among the eight mental health
 providers who participated in the first set of focus groups. A copy of the CCD-V manual will
 be given to each of these mental health providers. These reviewers will be asked to evaluate
 aims, structure, concerns, and other issues of acceptability of this intervention. After
 everyone independently reviews the manual, Dr. Mueller and I will set up two focus groups,
 which will include open-ended questions and general reactions to a short walk-through of the
 intervention. The purpose of these questions is to assess the perceived feasibility of the
 CCD-V intervention (as discussed by Bowen and colleagues). Specifically, our interview guide
 will explore perceived acceptability, demand, implementation, and integration of the proposed
 CCD-V intervention. Interviews will be audio recorded and will be transcribed by a VA
 approved contractor. These transcriptions will be analyzed using NVIVO. The same qualitative
 analytic approach previously discussed (Hamilton Rapid Turn-around) will be utilized for this
 qualitative inquiry. Any themes suggesting problems with acceptability, demand,
 implementation, and integration will be presented (via video conferencing) to the experts
 attending the two day manual development retreat. If consensus cannot be reached among the
 mentor and content expert about whether or not to make changes to the manual based on this
 qualitative inquiry, final decisions about treatment modifications will be based on the
 majority perspective.

 AIM 2: Pilot test the manual by treating a small number of Veterans (n=10) in an open trial.

 Sample/Recruitment/Screening. The sample will consist of 10, newly enrolling, Veterans in
 TWE. All Veterans who enter TWE will be informed about this voluntary research study. The
 study candidates will be invited to meet personally with a research assistant or the PI who
 will conduct the screening, thoroughly explain all procedures, and answer any questions
 before obtaining written informed consent on the consent form approved by Bedford VAMC
 Institutional Review Board (IRB). Any participant who is found to meet any exclusion criteria
 will not be eligible to participate in the study. The inclusion or exclusion of subjects will
 not be based on ethnic or racial characteristics. After screening and informed consent
 procedures are completed, the PI will begin meeting with the subject for weekly appointments
 following the CCD-V manual. Veterans participating in this phase of research will complete
 several surveys and questionnaires throughout the treatment.

 It is estimated that two Veterans per month will be recruited (total of 5 months to reach
 target sample of 10 subjects). During FY18, the Bedford VA enrolled 148 Veterans through TWE
 and 97.7% of these Veterans had a psychiatric diagnosis. Moreover, the average length of stay
 of Veterans in CWT programs was 419.6 days. Thus, recruitment of 10 subjects to participate
 in an eight-week treatment program, will be an achievable goal.

 Assessment Measures and Timeline. A variety of reliable and valid measures will be used in
 this phase to assess demographics, career development constructs, vocational behavior,
 employment functioning, working alliance, and client satisfaction.

 Qualitative Exit Interviews. At termination of treatment, participants will participate in a
 semi-structured, 30-minute exit interview with a research assistant who was not involved in
 the delivery of treatment. Questions for these qualitative exit interviews were derived from
 the items on the Client Satisfaction Questionnaire-8 (CSQ-8) - which will be completed by
 research participants at treatment termination - in order to provide more detailed
 information about the nature of participants experiences impacting satisfaction with
 treatment. The CSQ-8 measures satisfaction, perceived quality, and effectiveness of
 treatment. The following content areas will be assessed through these interviews
 (corresponding items from the CSQ-8 are listed but the items are not reproduced due to
 copyright): (1) general thoughts/experiences with treatment (CSQ-8 Question #1, 2, 3, 5); (2)
 perceived benefits of participating in treatment (CSQ-8 Questions #2, 3, 4, 6, 8); (3)
 specific changes and outcomes obtained because of participating in treatment (CSQ-8 Questions
 #6, 8); (4) perceived limitations of the treatment (CSQ-8 Questions #1, 4, 7, 8); (5)
 recommendations for enhancement/improvement of services (CSQ-8 Questions #1, 2, 5, 7, 8). We
 will also ask open-ended questions about usability of measures/assessments. Exit interviews
 will be audio recorded, for transcription and analysis.

 Data Management. Subject files will be kept in a locked cabinet with a locked office at the
 Bedford VA. Subjects will be given an ID number unrelated to any identifying information. An
 electronic, password protected file linking participants to their ID number will be kept on a
 VA computer on a secure, VA approved server. Data gathered through quantitative measures will
 be will be stored on a VA computer on a secure, VA approved server. All audiotaped interviews
 will be professionally transcribed by a VA approved contractor (see Budget Justification).
 These transcriptions will be stored on a VA computer on a secure, VA approved server.

 Data Analysis. The same qualitative analysis plan described in Aim 1 (the Hamilton Rapid
 Turn-around Analytic Approach) will be used for this aim. NVIVO will be used for analysis.
 Quantitative data gathered throughout the open trial will be used to triangulate findings
 from the qualitative exit interviews. A weaving approach will be used to integrate
 qualitative and quantitative findings through narratives on a theme-by-theme, or
 concept-by-concept basis. In this case, the themes/concepts will be based on the CSQ-8 (e.g.,
 satisfaction with treatment, perceived quality of treatment and effectiveness of the
 treatment). We will assess for fit between the data looking for confirmation of results and
 possible expansion of findings. Patterns of concern, challenges, dissatisfaction, or
 improvements found in integrated data will be reviewed by the PI and mentors, possibly
 resulting in treatment modification prior to pilot RCT in Aim 3. Discordance in qualitative
 and quantitative results will also be reviewed by the PI and the primary mentor team,
 possibly resulting in modification to the assessment of client satisfaction (i.e., CSQ-8 or
 Qualitative Exit Interview questions) prior to pilot RCT in Aim 3. Whether or not to modify
 treatment will be based on fidelity to best available research and feasibility of
 implementing recommended changes. If consensus cannot be obtained among the mentor team, then
 the majority perspective will be used to make final decisions. Patterns in incomplete or
 missing data, reported fatigues, or issues with the measurement paradigm in this Aim of the
 study will be addressed before implementation during pilot RCT in Aim 3.

 AIM 3: Conduct a randomized controlled pilot study comparing CCD-V (n=25) to treatment as
 usual (n=25).

 Sample/Recruitment/Screening. The sample will consist of 50, newly enrolling, Veterans in
 TWE. Only veterans entering TWE will be recruited for this study, currently enrolled TWE or
 SE veterans will not be eligible. Recruitment, screening, and inclusionary/exclusionary
 criteria are the same as Aim 2

 Study Procedures. Veterans who have completed screening and informed consent procedures will
 be randomized to one of two study arms: CCD-V (intervention) or treatment as usual (TAU;
 control). To ensure equal distribution across the intervention and control conditions, a
 permutated block randomization method will be utilized. Randomization will use
 computer-generated numbered lists; one for each condition (treatment and control). The lists
 will be generated through a block design where the block sizes are randomly selected among
 block sizes of either 4 or 6. There will be five rounds of each of the block sizes to
 randomize the 50 veterans into either the treatment or control condition. With this approach,
 every study arm assignment on the randomization list will consist of an even number of
 treatment or control assignments in random order. This approach has been discussed by
 previous researchers and not only guarantees balance in the two conditions, but also ensures
 that the investigator is not able to predict the treatment sequence as they will not know if
 the block size is 4 or 6. This procedure will result in 25 participants per condition which
 is in line with previous recommendations of a minimum of 15 subjects per condition. Subjects
 randomized to the treatment group will receive the full CCD-V treatment protocol, in addition
 to TWE (i.e., treatment as usual). Subjects randomized to the control group will receive
 treatment as usual (i.e., TWE). Subjects will be asked to attend a brief meeting with a
 researcher at a three-month follow-up. During this follow up meeting, participants will
 complete a variety of self-report questionnaires.

 Control Condition. Treatment as usual will serve as the control group for this aim of the
 study. Treatment as usual will consist of participation in Transitional Work Experience
 (TWE). Upon admission, veterans in TWE are quickly placed into a structured work setting.
 These time-limited work settings may be within the community, many are within the VA. CWT
 pays veterans through funds received by contracts from companies who contact TWE labor. There
 is minimal formal assessment or other vocational services prior to a veteran being placed
 into a work site. Work sites vary by TWE program. Common TWE placements at the VA include
 housekeeping, laundry, grounds maintenance, and food services. TWE program staff, which can
 range from peer support staff to masters trained vocational counselors, provide some on-site
 and off-site job coaching to help veterans maintain their TWE placement, in addition to some
 guidance on competitive job searching. TWE program staff tend to have (1.5 to 2 times larger
 caseloads that counselor in supported employment; approximately 25-40 veterans).

 Clinician Recruitment. The PI will recruit and train at least four master's-level trained
 clinicians, in the delivery of the CCD-V treatment protocol (two clinicians per 15 months).
 All clinicians will be employed at the Bedford VA.

 Treatment Adherence. There will be multiple fidelity checks to ensure treatment adherence
 among study clinicians. First, all CCD-V clinicians will participate in a didactic training
 introducing clinicians to the treatment protocol. Training of study clinicians is modeled
 after the VHA evidence-based practice (EBP) national training program. The national EBP
 training programs consist of an initial, three-day intensive didactic training. For this
 proposed study, Dr. Stevenson, as well as Drs. Mueller and Blustein, will lead a 9-hour
 workshop held over the course of three days (3 hours/day) for an initial didactic overview of
 the CCD-V protocol. Secondly, CCD-V clinicians will be required to attend weekly group
 consultation with the Dr. Stevenson (1 hour/week) for the first three months of providing
 treatment to address technical issues in the provision of the treatment. Thirdly, all CCD-V
 sessions will be video recorded, and these sessions will be reviewed and rated (on a variable
 time schedule) for treatment adherence by two staff researchers using the fidelity checklist
 developed in Phase One. Other researchers96 have noted that using a variable schedule to
 monitor treatment adherence among study clinicians can be an effective methodological
 approach to offset the time and resources needed to rate every-single session (in this
 proposed study there will be 25 Veterans assigned to the treatment condition, which consists
 of eight sessions, totaling 200 sessions). Once a session is rated, researchers will provide
 in-person, individual feedback to clinicians.

 Assessment Measures and Timeline. The same measurement paradigm described for Aim 2
 (screening, first session, last session) will be used for Aim 3 with the addition of a
 three-month follow up. At the three-month follow up Veterans will complete the same battery
 of measures completed at their final session. Qualitative interviews will be conducted and
 will be conducted as described in Aim 2, with some additional (new) questions regarding
 acceptability of randomization.

 Data Management. The same data management system described for Aim 2 will be used for Aim 3.

 Data Analysis. Consistent with the recommended stage model for development of behavioral
 therapies, the aim of this Stage I Pilot RCT is to inform future treatment development, and
 research and evaluation plans. To have .80 power to detect differences at a two-tailed alpha
 level of .05, a medium effect would require N=125. This is beyond the scope of a CDA-2 award
 and is more appropriate for a Stage II trial. A total sample size of 50 is feasible and
 consistent with recommendations of 15 to 30 subjects per condition for stage I behavioral
 treatment development. Outcomes gathered from this study will provide feasibility and
 acceptability data, as well as preliminary efficacy.

 Primary analyses will involve examination of the feasibility and acceptability of a RCT of
 CCD-V. Feasibility will be verified by adequate 1) recruitment rates (i.e., 2-3 Veterans per
 month), 2) >90% fidelity of clinician adherence to the treatment protocol, and 3) data
 retention (i.e., 70% of participants who attend at least 6 sessions of CCD-V. An acceptable
 level of attrition will be 30% in each condition. We will also examine the willingness of
 Veterans to be randomized with a goal of 80% of those approached who agree to randomization,
 the number of referrals from clinicians, ease of recruitment based on the average number of
 contacts needed to get consent and to schedule visits, the number of eligible participants
 from the pool of potential participants, and adherence and compliance to the treatment
 protocol. We will compare Veteran acceptability between groups by conducting an independent
 samples t-test on the CSQ-8. We will compare attrition rates (<15% vs. >15%) between the two
 groups using a chi square and logistic regression analysis. We will also obtain qualitative
 data about the feasibility and acceptability of the CCD-V treatment and randomization
 procedures to confirm and expand upon quantitative data. The Hamilton Rapid Turn-around
 analytic strategy (previously discussed in Aim 1 and 2) will be utilized for Aim 3
 qualitative analysis.

 While efficacy is not our major analytic aim, given the preliminary nature of this study, we
 will examine differences in measures of vocational identity, vocational functioning,
 psychiatric functioning, work importance and confidence, career adaptability, job search
 behavior, decent work, and job need satisfaction to identify empirical targets for further
 refinement of the treatment and inform the design of the future Merit Review. Based on the
 intent-to-treat principle, all participants randomized to treatment will be included in
 analyses. We will run descriptive statistics as well as internal reliability (Cronbach's
 alpha) for all measures, Pearson's r to look at relationships between variables, and Cohen's
 D for any continuous measures and Cramer's v for any categorical measures to look at effect
 size. We will also look at individual trajectories of response to gain better insight into
 which participants do particularly well (or poorly) in the two interventions. Effect sizes
 and 95% confidence intervals will be used as only one element to estimate power and sample
 size to help plan for a Merit Review study.
Inclusion Criteria:

 -  Presence of current psychiatric disorder, as indicated by an ICD-10 diagnosis
 encounter in the participants VA medical record within the last six months, inputted
 by a licensed mental health provider (psychiatrist, psychologist, licensed clinical
 social worker, psychiatric nurse practitioner). The following diagnosis will
 constitute a psychiatric disorder: PTSD, Depressive Disorders, Neurotic Disorders
 (e.g., Generalized Anxiety Disorder, Panic Disorder, Agoraphobia, Obsessive Compulsive
 Disorder), Affective Psychoses (e.g., Bipolar I, Major Depression), Schizophrenic
 Disorders (e.g., Schizophrenia, Schizoaffective Disorder), Substance Use Disorders
 (e.g., Alcohol, Opioid, Cannabis, Cocaine, Sedative, Hypnotic, or Anxiolytic, Other
 Stimulants)

 -  Individuals must self-identify as either unemployed, underemployed, or employed but
 functioning poorly at work secondary to mental illness or substance abuse. Unemployed
 will be operationalized as zero hours of paid work, underemployment will be
 operationalized as self-report of falling into one of five categories underemployment
 described by Feldman85

 -  overeducated,

 -  job-education mismatch,

 -  skill underutilization,

 -  hours underemployed, or

 -  pay underemployed. Functioning poorly on a job will be based on veteran's
 self-report that he or she believes to be functioning poorly in their job. These
 problems must have been present continuously during the three months preceding
 enrollment in this study.

 -  Ages 18 and over

 -  Planning to remain in the workforce for three years beyond the end of this study

 -  Competent to provide written informed consent

 -  Must be eligible for service through the Veterans Health Administration (VHA)

 Exclusion Criteria:

 -  Under the age of 18

 -  Acute suicide/homicide risk requiring treatment focused on safety to self or others.
 This will be based on self-report of current and active thoughts of harm to self or
 others with intent and plan that requires crisis stabilization in an acute psychiatric
 unit.

 -  Not otherwise eligible for CWT services

 -  Currently enrolled in SE or TWE services

 -  SCL-699 score greater than 26

 -  VR-12 PCS89 score greater than 56

 -  VR-12 MCS89 score greater than 56

 -  Not eligible for VHA services