U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • My Bibliography
  • Collections
  • Citation manager

Save citation to file

Email citation, add to collections.

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

Do We Report the Information That Is Necessary to Give Psychology Away? A Scoping Review of the Psychological Intervention Literature 2000-2018

Affiliations.

Psychologists are spending a considerable amount of time researching and developing interventions in hopes that their efforts can help to tackle some of society's pressing problems. Unfortunately, those hopes are often not realized-many interventions are developed and reported in journals but do not make their way into the broader world they were designed to change. One potential reason for this is that there may be a gap between the information reported in articles and the information others, such as practitioners, need to implement the findings. We explored this possibility in the current article. We conducted a scoping review to assess the extent to which the information needed for implementation is reported in psychological intervention articles. Results suggest psychological intervention articles report, at most, 64% of the information needed to implement interventions. We discuss the implications of this for both psychological theories and applying them in the world.

Keywords: implementation; metascience; psychological interventions; scaling; social cognition.

Similar articles

Publication types

LinkOut - more resources

Full text sources, other literature sources.

Miscellaneous

full text provider logo

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Effectiveness of psychological interventions to improve quality of life in people with long-term conditions: rapid systematic review of randomised controlled trials

BMC Psychology volume  6 , Article number:  11 ( 2018 ) Cite this article

20k Accesses

21 Citations

24 Altmetric

Metrics details

Long-term conditions may negatively impact multiple aspects of quality of life including physical functioning and mental wellbeing. The rapid systematic review aimed to examine the effectiveness of psychological interventions to improve quality of life in people with long-term conditions to inform future healthcare provision and research.

EBSCOhost and OVID were used to search four databases (PsychInfo, PBSC, Medline and Embase). Relevant papers were systematically extracted by one researcher using the predefined inclusion/exclusion criteria based on titles, abstracts, and full texts. Randomized controlled trial psychological interventions conducted between 2006 and February 2016 to directly target and assess people with long-term conditions in order to improve quality of life were included. Interventions without long-term condition populations, psychological intervention and/or patient-assessed quality of life were excluded.

From 2223 citations identified, 6 satisfied the inclusion/exclusion criteria. All 6 studies significantly improved at least one quality of life outcome immediately post-intervention. Significant quality of life improvements were maintained at 12-months follow-up in one out of two studies for each of the short- (0–3 months), medium- (3–12 months), and long-term (≥ 12 months) study duration categories.

Conclusions

All 6 psychological intervention studies significantly improved at least one quality of life outcome immediately post-intervention, with three out of six studies maintaining effects up to 12-months post-intervention. Future studies should seek to assess the efficacy of tailored psychological interventions using different formats, durations and facilitators to supplement healthcare provision and practice.

Peer Review reports

Long-term conditions (LTC) are complex physical health issues that last a year or longer and require ongoing care and support [ 1 ]. As LTC may be treated but not reversed, long-term care for patients and specialised rehabilitation training for staff is required to deal with the permanent and/or disabling nature of conditions [ 1 , 2 ]. As a consequence of increased exposure to risk factors, the likelihood of experiencing a LTC shows a linear increase with age, with those aged 75 years or older being up to five times more likely to experience a LTC than any other age group [ 1 , 3 , 4 ]. As the proportion of those aged 65 years or older in Europe is projected to increase from 15% in 2000 to 23.5% in 2030, a major and increasing challenge is faced by public health to not only target LTC symptoms, but also the associated increased rates of disability and reductions in both healthy and overall life expectancy [ 5 , 6 ]. Furthermore, due to LTC resulting from a combination of genetic, physiological, psychological and socio-economic factors, LTC are also becoming increasingly prevalent in younger populations [ 6 ].

LTC encompass a wide range of conditions which impact upon one’s physical, psychological, and social functioning. However, as individual LTC may differ in aetiology, presentation and consequence, there is significant variability in the degree to which each LTC is medically understood, diagnosed and treated [ 1 , 6 , 7 ]. For example, cardiovascular disease and diabetes mellitus are two of the most prevalent and increasingly occurring LTC worldwide, and are associated with increased rates of long-term disability, dependency on others for everyday functioning, and depression [ 6 , 8 , 9 , 10 ]. Chronic obstructive pulmonary disease and dementia are prevalent but under-diagnosed LTC as symptoms may often be mistakenly attributed to an anticipated gradual age-related decline in functioning. However, both conditions relate to increased medical admissions, distressing symptoms, mortality, and disability [ 6 , 11 , 12 , 13 ]. Medically unexplained physical symptoms (MUPS) – such as chronic fatigue syndrome, irritable bowel syndrome and fibromyalgia – are also LTC that (despite having unknown aetiologies) profoundly impact psychological, emotional and physical functioning, as well as healthcare costs and requirements [ 14 – 16 ]. Furthermore, aforementioned conditions only provide a snapshot of overall LTC types, and disorder-related fatalities are also predicted to increase for manageable conditions such as asthma without further public health intervention [ 6 ].

While it is important to understand the causes, presentations, and consequences of LTC in isolation, to effectively understand the burden of LTC it is critical to look at how multiple LTC may co-occur and interact. While the terms ‘ Multi-morbidity ’ and ‘ Co-morbidity ’ are often used interchangeably, the former refers to several LTC coexisting, while the latter refers to multiple disorders stemming from one predominant LTC [ 17 , 18 ]. Effective determination of the worldwide rates of specific and multi-morbid LTC is complex because of issues with insufficient or inappropriate health measures and analyses being used, and between-country differences in LTC definitions and inclusion criteria [ 19 , 20 ]. However, regardless of the figures assessed, LTC pose a key challenge as 14–29% of the European population report one LTC and 7–18% report two or more conditions [ 21 ]. Furthermore, these conservative estimates consider a limited range of conditions, and when a broader range of LTC is considered these figures may be considerably higher. For example, 27% of 75–84 year olds in Scotland experience two or more LTC [ 1 ]. Hence, policy and interventions must not only target specific LTC, but also account for the often multi-morbid nature of LTC.

Health status is an effective measure of healthcare and intervention effectiveness; however, using solely population-level mortality and morbidity rates may be problematic as they only provide a snapshot of effects [ 22 ]. As a consequence, subjective measures such as quality of life (QOL), health-related QOL (HR-QOL) and mental wellbeing (MWB) are increasingly being used in healthcare research to assess subjective health status and condition-related burden and coping [ 22 ]. QOL is a multi-dimensional concept that includes subjective evaluations of one’s physical, psychological, emotional, social, functional and/or environmental state. Due to the wide range of potential constructs, QOL may be assessed using uni-dimensional, multi-dimensional, and individual measures [ 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 ]. HR-QOL and MWB are sub-domains of QOL that may be assessed using general or specific measures [ 23 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ]. HR-QOL relates to one’s perception of physical and mental health and may provide a valuable insight into symptomology–psychology links, while MWB relates to one’s ability to cope with life stressors and maintain a healthy mental state which may provide an insight into illness and coping perceptions [ 23 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ].

LTC diagnosis, treatment, and outcomes not only have a significant impact upon patients’ physical functioning, but may also have profound consequences for psychological wellbeing and QOL through affecting emotional, physiological and MWB. This may consequently impact upon medical outcomes through treatment choice and the likelihood of LTC relapse and survival [ 44 , 45 , 46 , 47 , 48 , 49 , 50 ]. Co-morbid mental health disorders are a key issue in LTC populations [ 11 ], with LTC patients being significantly more likely to be diagnosed with depressive and/or anxiety disorders [ 51 , 52 ]. This may relate to poorer health outcomes and self-care, more severe symptoms, reduced medical adherence, and increased unhealthy behaviours, healthcare spending, and disorder-related death rates [ 51 , 52 ]. Despite this, traditional medical models often overlook key psychological variables through employing a paternalistic care approach where clinicians exercise predominant authority over patients’ care [ 53 , 54 , 55 ]. Therefore, as LTC outcomes not only relate to healthcare treatment but are also intrinsically linked to psychological wellbeing and mental health, the provision of psychological interventions and therapies is critical for LTC healthcare services and patient outcomes [ 11 , 56 , 57 ].

Previous systematic reviews (SR) have demonstrated efficacy for psychological interventions (provided in a wide range of formats) to improve both QOL and physical health outcomes in specific LTC patients. For example, mindfulness for multiple sclerosis and cancer, psychosocial interventions for diabetes and cancer, cognitive behavioural therapy (CBT) and relaxation for recurrent headaches, and internet-based CBT or coaching for chronic somatic conditions [ 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ]. However, to the researchers’ knowledge, there has not previously been a SR that attempts to only assess studies with high scientific rigour that utilise psychological interventions across LTC in order to provide valid comparisons for the effectiveness of interventions and guide LTC healthcare development. As aforementioned, as research has demonstrated that LTC may have profound physiological and psychological effects [ 1 , 6 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 ], rates of specific and multi-morbid LTC are high and predicted to rise [ 3 , 4 , 5 , 6 , 17 , 18 , 21 ], and psychological interventions may improve both QOL and physical functioning [ 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 ], it is crucial to determine which interventions may be effective across conditions.

The rapid SR aimed to examine the effectiveness of a variety of psychological interventions that seek to improve generic or specific QOL, HR-QOL and/or MWB in people with LTC to determine whether specific interventions may be viable and efficacious for general LTC healthcare implementation. As randomised controlled trial (RCT) designs are the most rigorous and effective method for determining whether intervention–outcome relationships are present [ 67 ], and to ensure valid comparisons were possible between studies, only RCTs with a usual care control (UCC) condition which directly target and assess patients with a current LTC diagnosis were included. To ensure the review assessed the most up-to-date research, only studies published between 2006 and February 2016 were included. Furthermore, despite a general dose and duration effect being present for psychological intervention effectiveness, evidence relating to the optimum duration of psychological interventions for LTC to achieve maximum effectiveness is mixed [ 62 , 68 , 69 ]. Therefore, an ante hoc decision was taken to categorise studies by intervention facilitation duration, encompassing short- (0–3 months), medium- (3–12 months) and long-term (≥12 months) study classifications.

Rapid systematic review

Rapid SR are a form of streamlined SR that may be used by healthcare professionals to guide policy in a time-frame that may not be possible using traditional SR methods. While they do not provide as in-depth information and should not be viewed as a substitute for traditional SRs, rapid SR may have important implications for healthcare decision-making through using systematic methods to provide high-quality information and draw significantly similar conclusions to a traditional SR [ 70 , 71 , 72 ]. As the review was conducted during NHS employment and aimed to influence healthcare policy, utilizing a SR procedure was deemed the most feasible and practical approach based on two key considerations. First, in order for the research to have implications (not only for research but also) for healthcare, it was critical that high quality information was provided using limited time and resources [ 70 ]. Second, as the research was conducted during NA’s NHS employment as one competency of a two-year professional doctorate-level Health Psychology qualification, the ability to generate a complete draft of findings for NHS stakeholders within a maximum of 6 months (as opposed to up to 2 years for a traditional SR) [ 70 , 71 , 72 ] was deemed the most appropriate approach. Therefore, two researchers (NA, GO) followed traditional SR procedures but without searching grey literature and with only one researcher (NA) involved until data extraction was completed. The implications of adopting this approach are presented in ‘Rapid Systematic Review Strengths and Limitations’ .

Search strategy, selection criteria and data extraction

Searches were conducted on 19.02.2016 by one researcher (NA) using EBSCOhost to access PsychInfo (1967–2016) and PBSC (1974–2016), and OVID to access Medline (1946–2016) and Embase (1974–2016). Both databases were searched using key terms (Table  1 ), with potential citations suitability assessed using the pre-defined inclusion/exclusion criteria (Table 2 ). Due to the multi-dimensional nature of QOL there is currently no universally accepted definition of QOL [ 22 , 25 ]. Therefore, an ante hoc decision was made to manually assess individual studies for the presence or absence of QOL rather than include it in the search terms. Additionally, only RCTs with a UCC were included in order to ensure that valid comparisons of rigour and effectiveness were possible between different interventions and LTC [ 67 ]. Data were extracted using a template developed from the COCHRANE criteria [ 73 ]. As the SR aimed to guide public health policy, the Effective Public Health Practice Project (EPHPP) ‘ Quality Assessment Tool for Quantitative Studies ’ was used to assess study quality [ 74 ].

Study selection

The PRISMA flowchart (Fig.  1 ) demonstrates the process used to narrow 2224 prospective citations to 13 studies based on titles and abstracts [ 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 , 87 ], with 6 studies satisfying the inclusion/exclusion criteria based on full articles [ 82 , 83 , 84 , 85 , 86 , 87 ].

Study Selection Process

Study characteristics

Key study features, measures, results (including significance values and effect sizes where stated), and authors’ conclusions from the 6 eligible studies are presented in Table 3 . The six studies [ 82 , 83 , 84 , 85 , 86 , 87 ] encompass a variety of psychological interventions and durations: 2 were short-term (0–3 months) [ 82 , 85 ], 2 were medium-term (3–12 months) [ 84 , 86 ], and 2 were long-term studies (≥12 months) [ 83 , 87 ]. Facilitators of the interventions varied considerably between studies, with nurses facilitating 3 interventions [ 83 , 85 , 87 ], and the remaining 3 studies being facilitated by health educators [ 82 ], CBT therapists [ 84 ], and clinical psychologists [ 86 ]. Additionally, each intervention focussed on a different LTC; comprising asthma [ 82 ], human immunodeficiency virus (HIV) [ 83 ], MUPS [ 84 ], congestive heart failure (CHF) [ 85 ], knee osteoarthritis [ 86 ], and head & neck cancer (HNC) patients [ 87 ]. Five studies compared a UCC with one intervention [ 82 , 83 , 84 , 85 , 87 ], while one study contrasted multiple interventions with a UCC [ 86 ]. Furthermore, all 6 studies comprised samples of both genders aged 18 years or over, and assessed (among other measures) generic and/or specific measures of QOL, HR-QOL and/or MWB [ 82 , 83 , 84 , 85 , 86 , 87 ].

Study quality assessment

EPHPP quality assessment [ 74 ] involves assessing studies based on 6 key components (Table  4 ). Each component comprises multiple choice questions for which scores are combined to provide an overall component rating of ‘ Strong ’, ‘ Moderate ’ or ‘ Weak ’. All component ratings are then combined to provide an overall quality rating of ‘ Strong ’ for no ‘ Weak ’ components, ‘ Moderate ’ for one ‘ Weak ’ component, and ‘ Weak ’ for two or more ‘ Weak ’ components.

Short-term interventions (0–3 months)

Two short-term interventions were present. Baptist et al. [ 82 ] offered a 6-week health educator-led self-regulation intervention for asthmatic patients ( N  = 70), comprising 3 consecutive weekly health education group sessions followed by 3 weekly one-to-one telephone sessions. Health educators received a 2-day training session on self-regulation and asthma management principles which was used to conduct tailored self-regulation interventions. This involved patients’ self-selecting a specific asthma-related problem that they wished to address before planning how to achieve positive outcomes and cope with potential asthma-related issues. Significant improvements were present 12-months post-intervention for overall asthma-related QOL, activity, control and hospitalisations. QOL symptom and environment improvements were present 1-month post-intervention, and non-significant changes occurred for QOL emotions or emergency department usage.

Smeulders et al. [ 85 ] offered a 6-week, 150-min per week structured self-management programme for CHF patients ( n  = 317). The intervention was co-facilitated by a cardiac nurse specialist and a CHF patient (acting as a peer role model) who were both trained on a 4-day ‘ Chronic Disease Self-Management Programme ’ [ 88 ] by a research and CHF nurse specialist. This incorporated four strategies to enhance self-efficacy over one’s condition: skills mastery, behaviour modelling, social persuasion and symptom reinterpretation. Significant improvements were present immediately (but not at 6- or 12-months) post-intervention for cardiac-specific QOL, cognitive symptom management and self-care behaviour. However, non-significant intervention effects were present at all time-points for perceived control, general self-efficacy, and all other QOL outcomes (general QOL, perceived autonomy, and anxiety and depression).

Medium-term interventions (3–12 months)

Two medium-term interventions were present. Escobar et al . [ 84 ] offered 10, 45–60-min CBT therapist-led sessions over a 3-month period to MUPS patients ( n  = 172). Two therapists received training from two authors employed by Departments of Psychology and Psychiatry respectively, with protocol adherence routinely evaluated using “taped” recordings. Key topics included managing physical distress, relaxation, activity regulation, emotional awareness, cognitive restructuring and interpersonal communication. The intervention significantly improved patient-rated depression and current somatic symptoms, and physician-rated global severity of symptoms, immediately post-intervention. Only changes to patient-rated somatic symptoms were maintained 6-months post-intervention and no effects were present for anxiety or physical functioning.

Somers et al. [ 86 ] ‘ Pain Coping Skills Training ’ (PCST) and ‘ Behavioural Weight Management ’ (BWM) co-interventions for knee osteoarthritis patients ( n  = 232) were conducted by clinical psychologists (with 1–6 years experience in their respective area), under the supervision and training of an experienced senior clinical psychologist. The intervention spanned 24 weeks, comprising 12 weekly groups sessions followed by 12 weeks of sessions every second week for the remainder of the intervention. One group received BWM based on the ' LEARN ' programme [ 89 ], which focused on lifestyle, exercise, attitudes, relationships and nutrition. The second group received PCST, which focused on maladaptive pain catastrophizing and adaptive coping strategies. The third group received both BWM and PCST programmes. While the study did not utilise a generic measure of QOL, the combined intervention demonstrated significant improvements compared to UCC 12-months post-intervention for arthritis- and weight-specific self-efficacy, pain symptoms and catastrophizing, physical disability and stiffness, weight, and BMI.

Long-term interventions (≥12 months)

Two long-term interventions were present. Blank et al. [ 83 ] offered weekly community-based psycho-education and symptom management sessions (of unspecified duration) over a 12-month period to HIV patients ( n  = 238). Four Advanced Practice Nurses facilitated psycho-education sessions for coping with barriers and self-care, and provided resources to support patients’ to organise their medication regimens. In addition, the Practice Nurses coordinated a multi-disciplinary team of physical and mental healthcare providers to provide tailored medical and mental healthcare. Growth curve analyses were used to assess outcomes, demonstrating significant improvements 12-months post-intervention for the HR-QOL mental health subscale and viral load. However, non-significant improvements were present for the HR-QOL physical health subscale and immune functioning.

Van Der Meulen et al. [ 87 ] offered six bimonthly 45-min nurse-led, problem-focused counselling sessions for depressive symptoms to HNC patients ( n  = 205) over a 12-month period. Three experienced oncology nurses received a one-day training course from two psychologists and one investigator on the ‘ Nurse Counselling and After Intervention ’. Session recordings were reviewed every 2 months to assess intervention quality. The intervention focussed on managing the physical, psychological and social consequences of HNC, restructuring illness cognitions and beliefs, education and behavioural relaxation training, and providing emotional support. Significant improvements were present immediately post-intervention (both in the overall sample and depressive subgroup) for the primary endpoint of depressive symptoms and secondary endpoint of overall physical symptoms.

General statement

The review aimed to examine the effectiveness of psychological interventions to improve specific or generic components of QOL, HR-QOL and/or MWb in people with LTC, with a view to advising LTC healthcare provision. The findings, strengths, limitations and implications of studies, and the strengths and limitations of the current review and rapid SR procedure, are discussed.

Six-week self-regulation for older adult asthmatics

Baptist et al. [ 82 ] trained health educators on a two-day programme which enabled them to facilitate a six-week self-regulation intervention. As a consequence of the self-regulation intervention, significant improvements occurred for older adults’ overall asthma-related QOL and control up to 12-months post-intervention. The key hallmarks of the self-regulation approach was to facilitate patients’ self-identification of a specific condition-related issue and potential barriers and goals, in order to provide tailored support and increase patients’ self-efficacy over their condition. This approach has also been used to achieve positive outcomes for heart disease and medical noncompliance in older adults [ 90 , 91 ]. Therefore, when combined with the low attrition rate (7%) [ 82 ] and self-regulation concepts not being unique to asthma [ 92 ], self-regulation provides promise as an effective and acceptable form of intervention to improve QOL in older adults. Despite receiving ‘ Strong ’ ratings for all but one quality component, the study received a ‘ Weak ’ ‘ Selection Bias ’ rating due to only 54% of those approached agreeing to participate, which may have two potential implications. First, this may indicate a lack of interest in self-regulation interventions potentially due to this approach differing from anticipated traditional asthma care approaches [ 82 ]. Second, while double-blinding improves methodological quality [ 93 ], a lack of awareness of intervention procedures and potential benefits may have impact enrolment. Additionally, as highlighted by the authors, the study was limited by using a single site and required a certain threshold of patient communicative ability to contribute to group discussions. Therefore, while additional studies and a cost-benefit analysis would be required to determine the efficacy of larger scale programmes, and consideration is required for the enrolment confounds, the study demonstrated that a short-term, health educator-led self-regulation intervention may have promising implications for LTC healthcare.

Six-week structured self-management for CHF

Smeulders et al.’s [ 85 ] 6-week structured self-management intervention, co-facilitated by a trained cardiac nurse specialist and a CHF peer role model, significantly improved cardiac-specific QOL immediately post-intervention. However, effects were not maintained at 6- or 12-months follow-up, and no other QOL improvements occurred. Despite having four ‘ Strong ’ components, the study received an overall ‘ Weak ’ EPHPP quality rating due to unspecified ‘ Blinding ’ of patients and clinicians, and a ‘ Selection Bias ’ as only 44% of eligible patients participated. As justification for non-participation varied considerably – from a lack of interest to physical, psychosocial or cognitive problems preventing participation – a qualitative study to further explore enrolment issues may be beneficial to determine whether the intervention was sufficiently tailored to complex CHF needs. While the authors proposed that non-significant effects may have resulted from insufficient intervention length or intensity above the “relatively high level” of Dutch standard care, a similar medium-term (15 weeks) self-management intervention improved physical but not emotional QOL [ 94 ]. Therefore, despite positive short-term results, further research is required to understand the mechanisms behind the low participation and lack of long-term QOL effects for structured self-management, with a view to using this to develop and trial more tailored interventions.

Overall short-term interventions

Despite both short-term interventions reviewed [ 82 , 85 ] comprising 6-week programmes, considerable differences were present between-interventions that may have influenced outcomes. First, the self-regulation intervention was solely facilitated by health educators, while the CHF intervention was co-facilitated by a nurse and a patient ‘ peer leader ’. While peer leaders were trained to effectively facilitate the intervention, potential differences in pre-existing knowledge and experience associated with not being a trained healthcare professional may have influenced the content, approach and style of programme adopted, and subsequently QOL outcomes. Second, research into the mechanisms behind why the 2-day (but not the 4-day) training resulted in significant long-term QOL improvement would be beneficial. Three possible explanations for this include potential differences in the quality of training, that health educators may benefit more from short-term training than nurses and/or peer leaders, and/or that additional information provided during the longer training may have resulted in a more structured but less tailored approach being adopted with patients. Third, as asthma and CHF differ considerably in emotional, physical and social outcomes [ 95 , 96 ], this may have impacted the long-term maintenance of intervention effects post-intervention and consequently QOL outcomes. Fourth, methodological differences may have impacted outcomes due to the discrepancy between Blank et al.’s [ 82 ] ‘ Moderate ’ and Smuelders et al.’s [ 85 ] ‘ Weak ’ EPHPP quality ratings. However, despite considerable differences, both studies demonstrated that interventions which actively engage and involve the patient in their care may significantly improve at least short-term QOL, and that, while achieving initial buy-in for these types of interventions may be challenging, once enrolled attrition rates were low. Therefore, while cost-benefit analyses and further research are required to determine viability and overcome current limitations, short-term psychological interventions that actively involve patients demonstrated initial promise for improving QOL, with self-regulation demonstrating particular promise.

Three-month CBT for medically unexplained symptoms

Escobar et al.’s [ 84 ] structured CBT therapist-led intervention for MUPS significantly improved patient-rated depression and somatic symptoms, and clinician-rated severity of symptoms, immediately post-intervention. However, only improvements to patient-rated somatic symptoms were maintained 6-months post-intervention. While depressive and somatic symptom improvements were anticipated as CBT is widely advocated for depression, the improvements in both patient- and clinician-rated MUPS symptoms potentially indicate additional benefits for short-term perceived behavioural and cognitive control. Despite positive results, achieving patient buy-in was problematic as only 41% of eligible patients enrolled with an attrition rate of 45%. While the justification for this was not discussed, the study proposed that future programmes may benefit from using a staged-approach to tailor the intervention to patients’ needs, use of other services, costs, and the delivery setting. As MUPS patients do not benefit from reassurance alone [ 97 ] and a similar 6-week CBT programme for Breast Cancer patients demonstrated non-significant results [ 98 ], this highlights the need for at least moderate-length, tailored CBT-based interventions that are tailored to patients’ needs. Therefore, while research is required to overcome the confounds of participation and long-term effect maintenance, and to determine how to feasibly implement the complex and time-consuming intervention in practice, CBT demonstrated promise for improving QOL in LTC.

Six-month BWM/PCST for knee osteoarthritis

Somers et al.’s [ 86 ] clinical psychologist-led 24-week combined PCST and BWM intervention demonstrated significant improvements 12-months post-intervention for the QOL components of arthritis- and weight-specific self-efficacy, pain symptoms and catastrophizing, physical disability and stiffness, weight, and BMI compared to UCC. Additionally, the combined intervention was significantly more effective than the individual interventions for the aforementioned outcomes; excluding PCST for pain catastrophizing and one pain measure. This demonstrates that by conducting a programme which not only targets LTCs’ physical components, but also enables people to cope with the psychological effects and consequences, significantly improves both physical and psychological QOL. However, despite being one of only two studies reviewed to receive a ‘ Strong ’ quality rating, the study was confounded by the combined condition receiving double the intervention dosage than individual conditions. Additionally, as interventions were facilitated by highly trained clinical psychologists, additional research and a cost-benefit analysis comparing this approach with training existing staff involved in arthritis healthcare to provide the intervention would be beneficial. Therefore, while research for potential dose and expertise effects is required, the study demonstrated efficacy for a medium-term intervention to improve QOL 12-months post-intervention through targeting both the physical and psychological components of LTC.

Overall medium-term interventions

Overall, the medium-term studies [ 84 , 86 ] demonstrated effectiveness for interventions delivered by psychologically trained staff to improve QOL in LTC, with CBT resulting in short-term improvements and a combined physical and psychological intervention resulting in improvements 12-months post-intervention. While these studies highlighted the need for medium-term psychological interventions to be tailored to LTC patients’ physical and psychological needs in order to actively involve patients in their healthcare, three considerations are required. First, differences were present in the quality of studies, with Escobar et al. [ 84 ] receiving a ‘ Moderate ’ quality rating and Somers et al. [ 86 ] a ‘ Strong ’ rating. As this stemmed purely from the CBT-therapist intervention experiencing more problematic ‘ Withdrawals & Dropouts ’ [ 84 ], future research into the mechanisms behind this difference would be beneficial. Second, despite both LTC having profound physical and psychological consequences, current understanding of the causes and consequences of MUPS is less well defined than for knee osteoarthritis, which may have impacted outcomes [ 84 , 86 ]. Third, while the positive outcomes provide an important foundation for research to build upon, consideration is required for the level of staff input and training required to conduct such programmes. As becoming a chartered psychologist or CBT therapist typically takes at least 6–7 years of study and training in addition to vocational work, both programmes required highly specialised staff. While this appears beneficial for QOL outcomes, this raises potential practicality issues for healthcare implementation as considerations would be required to determine capacity, practicality and financial viability within existing or additional services. However, as Somers et al. [ 86 ] demonstrated greater improvements based on psychological intervention dosage, this highlights a potential opportunity to utilise psychological principles to improve QOL outcomes for LTC. Therefore, careful consideration is required for the implementation of medium-term interventions using psychologically trained staff; however, the positive effects for both physical and psychological QOL indicate promise for healthcare.

Twelve-month psycho-education and management for HIV

Blank et al.’s [ 83 ] 12-month, nurse-led community-based psycho-education and healthcare management intervention for HIV patients demonstrated significant improvements for mental health QOL and immune functioning 12-months post-intervention. However, no effect was present for physical health QOL or viral load. The rationale behind the study was that reforms to healthcare provide a challenge but also an opportunity to redesign systems in a more integrated manner. Through training nurses to facilitate psycho-education while providing tailored access to relevant professions within a multi-disciplinary healthcare team, significant improvements were present for condition-related immune functioning and mental health. However, future healthcare research would benefit from factoring in key confounds. First, as university-based nurses facilitated the intervention the additional research experience associated with this work setting may have influenced outcomes. Second, as viral load changes only occurred 12-months post-intervention, consideration of optimal intervention and assessment duration is required. Finally, while assessing different constructs at different time points may be the most feasible approach within multi-disciplinary interventions, careful consideration is required for the effect this may have on analyses and attrition, as 75% of patients completed the QOL measure 12-months post-intervention compared with only 61% providing bio-markers data. Therefore, while future work may benefit from overcoming practical confounds, altering existing services to provide psycho-education and tailored management of a multi-disciplinary team by nurses may be a feasible, cost-effective approach.

Twelve-month counselling for HNC

Van Der Meulen et al.’s [ 87 ] 12-month, nurse-led problem-focussed counselling programme significantly improved depressive and physical symptoms in HNC patients immediately post-intervention, with effects being more pronounced in the depressive-subgroup. As the authors proposed that those with greatest physical impairments were more likely to experience depressive symptoms and those with depressive symptoms benefited most from the intervention, problem-focussed counselling demonstrated efficacy both for the general sample and for those patients in greatest need. While the study was confounded by a ‘ Moderate ’ ‘ Selection Bias ’ with only 63% of eligible patients participating, it was one of only two studies to receive a ‘ Strong ’ overall rating and once enrolled attrition rates were low (13%). Therefore, as low attrition supports the authors’ claim that utilising nurse facilitators may not only reduce healthcare costs but also stigma, the intervention was feasible and cost-effective. Hence, due to the positive intervention effects a combined with the psychological elements of the interventions not being specific to HNC, theory-based long-term nurse-facilitated interventions provide promise for LTC healthcare delivery.

Overall long-term interventions

Overall, the long-term studies [ 83 , 87 ] demonstrated efficacy for long-term nurse-led interventions to improve QOL in LTC, with HNC counselling having significant post-intervention effects, and HIV psycho-education and care management improving QOL 12-months post-intervention. Despite differences in the format, content and delivery of interventions, significant QOL improvements were achieved through supporting nurses to facilitate interventions that enabled patients to develop the skills, knowledge and efficacy required to manage the physical and psychological components and consequences of their LTC. Furthermore, as both HIV and HNC are complex LTC that may have profound physical and mental effects and therefore require a large amount of medical support, the positive intervention effects provide promise for other complex LTC. As proposed by Van Der Meulen et al. [ 87 ], utilising nurses to provide long-term interventions may be both a financially and practically viable approach to implementing long-term psychological interventions, and may reduce stigma due to nurses already being intrinsically involved in LTC healthcare provision. However, consideration is require for the differences between Blank et al.’s [ 83 ] ‘ Moderate ’ and Van Der Meulen et al.’s [ 87 ] ‘ Strong ’ quality ratings, with this stemming from the ‘ Weak ' and ‘ Strong ’ ‘ Withdrawals & Dropouts ’ quality ratings respectively. Therefore, future research is required into the mechanisms behind between-study differences in enrolment and attrition despite both interventions utilizing nurse facilitators. Hence, long-term, nurse-led interventions which actively involve patients in their care and target both the physical and psychological constructs of LTC provide promise for healthcare. However, further research is required to determine the optimal approach to adopt in order to enhance patient enrolment for such programmes.

General discussion

Implications of findings.

The studies reviewed demonstrated that psychological interventions for LTC varied considerably in terms of duration, population, methods, quality ratings, facilitators and long-term effectiveness. Descriptive analysis of findings indicated that all interventions resulted in significant improvements to at least one component of QOL immediately-post intervention. Furthermore, the 6-week health educator self-regulation intervention for asthma [ 82 ], 6-month clinical psychologist-led combined PCST-BWM intervention for knee osteoarthritis [ 86 ], and 12-month nurse-led psycho-education and care management intervention for HIV [ 83 ] significantly improved QOL 12-months post-intervention. While further research is required to assess the mechanisms behind differences in the effectiveness of interventions and the feasibility of implementing interventions in LTC healthcare, the findings indicate that psychological interventions utilising different formats, durations and facilitators which actively involve and enable patients to have self-efficacy over their care may result in significant QOL improvements for LTC patients.

In addition to the effectiveness of interventions, the studies have important implications for future research and healthcare. First, across studies enrolment in psychological interventions was low, with one study only successfully enrolling 41% of potential patients [ 84 ]. While blinding was often used to increase methodological quality, this may have influenced participation rates through blinding patients to the potential components, goals and benefits of interventions. Additionally, at present LTC treatments typically promote pharmacological and/or medical treatments, with psychological interventions promoted as secondarily [ 1 , 2 , 3 , 5 , 6 , 7 , 11 , 19 , 22 , 23 , 26 ]. This may promote patients to seek quick-fix treatments and requires a change in approach in order to enhance participation in psychological interventions. Further, as only 6 RCTs from 2006 to February2016 were deemed suitable based on the inclusion/exclusion criteria, coupled with the review demonstrating that psychological interventions may improve QOL across LTC, this review highlights the need for high-quality research into this area and the application of methods in healthcare. Hence, future research and interventions across LTC that attempt to build upon the positive findings and resolve methodological confounds is recommended in order to build a greater evidence base for the effectiveness of psychological interventions on LTC.

General Strengths and Limitations

Many of the strengths of the review may also be regarded as limitations. First, an ante hoc decision was made to include only RCTs with a UCC in order to ensure that only high methodological quality studies were included and valid comparisons could be made between interventions despite considerable differences in the LTC targeted [ 67 ]. Furthermore, in order to ensure that only the most up-to-date research was assessed, only studies spanning the previous 10 years (2006 to February 2016) were included. While discussions were conducted with relevant experts (within Public Health, Health Psychology and Publishing) prior to the review to set a strict inclusion/exclusion criterion for only the most relevant research, it is possible that important and interesting studies, findings and interventions may have been excluded. Additionally, in order to improve the reliability of findings, only studies that directly targeted LTC patients for both the intervention and assessment were included. However, this may also have reduced the number of interventions through excluding those that indirectly target or assess patients through clinicians, carers or family members, such as communicative or learning disorder populations who may benefit from psychological interventions but are unable to communicate effects. Finally, while he COCHRANE data extraction framework is well validated and used across disciplines [ 73 ], the EPHPP quality assessment tool was used as the review aimed to guide public health policy [ 74 ]. However, as the review assessed psychological interventions, alternative tools may potentially have been more appropriate and may have resulted in different quality ratings. For example, Smeulders et al. [ 85 ] received a ‘ Weak ’ rating despite demonstrating four ‘ Strong ’ components, and Van Der Meulen et al. [ 87 ] received a ‘ Strong ’ rating despite only stating significance values as ‘ p ≤ 0.05 ’. Therefore, future replications and expansions should attempt to build upon the strengths, and generate solutions for the limitations, of the review in order to improve upon the quality of the review.

Rapid systematic review strengths and limitations

Previous research has discussed the relative strengths and limitations of the rapid SR approach compared to traditional SRs [ 70 , 71 , 72 ]. One primary benefit of this methodology is that it may be used to assess research and formulate conclusions that influence healthcare policy within a time-frame and budget that would not be possible using traditional methods. While significant work was subsequently conducted to improve the review to publication standard, this methodology allowed the review to progress from defining potential search parameters to providing a first draft to healthcare stakeholders within three-months. Rapid SRs may potentially suffer from using a non-iterative search strategy, narrow time-frame for retrieval, not performing quality analysis, and limiting consultation with experts. However, the present review did not suffer from these confounds as a strict ante hoc criteria was set and adhered to, and various contacts (Public Health, Health Psychology etc.) were sought out to discuss the suitability of the review. Therefore, active efforts were made to strengthen methodology by ensuring that many potential confounds of rapid SRs were accounted for.

Despite attempts to maintain as high quality methodology as possible, implicit limitations are associated with one researcher being involved until data extraction. First, practical constraints meant that grey literature, reference lists and additional databases were not searched, which may have provided additional findings. Furthermore, while all possible effort were made to maintain accuracy, ‘ human error ’ and ‘ selection bias ’ are possible, and as only articles published in English were included ‘ publication ’ and ‘ language ’ biases are also possible. However, given the relative strengths and weaknesses of rapid SRs, and that the review was completed during NHS employment ( See Authors’ Information ), overall utilising rapid SR methodology was useful for an initial study. Therefore, future attempts should be made to replicate and expand upon the findings using a larger research team to limit the aforementioned confounds through continuing to utilise a strict ante hoc criteria.

The studies reviewed demonstrated promising results for utilising psychological interventions to improve QOL in LTC patients, with short-, medium- and long-term interventions that promote patient involvement demonstrating positive outcomes. While confounds were present which require resolution, particularly with low participation from eligible patients, the positive results indicated that with high-quality methodology, actively involving patients in their care and tailoring of interventions to patients’ needs, psychological interventions may improve QOL in LTC. Hence, future studies should assess the efficacy of tailored interventions utilising different formats, durations, and facilitators to improve QOL in LTC, while the development and promotion of services should be promoted to utilise psychological interventions to supplement medical care,

Abbreviations

Behavioural Weight Management.

Cognitive Behavioural Therapy

Congestive Heart Failure

Effective Public Health Practice Project

Human Immunodeficiency Virus

Head & Neck Cancer

Health-Related Quality of Life

Long-Term Conditions

Medically Unexplained Physical Symptoms

Mental Wellbeing

Pain Coping Skills Training

Quality of Life

Randomised Controlled Trial

Systematic Review

Usual Care Control

Burns H. Improving the health and wellbeing of people with long term conditions: A national action plan. Long Term Conditions Action plan. Scottish Government; 2009. http://www.gov.scot/Resource/Doc/294270/0090939.pdf . Accessed 17 Oct 2016

Timmerick TC. Dictionary of health services management. 2nd ed. Owings Mills: Maryland; 1987.

Google Scholar  

Gray L, Leyland A. Volume 1 main report. In: Scottish Health Survey 2012. Scottish Government. 2013. http://www.gov.scot/Resource/0043/00434590.pdf . Accessed 10.02.2016.

Janssen F. Cohort patterns in mortality trends among elderly in seven European countries, 1950-99. Int J Epidemiol. 2005;34(5):1149–59.

Article   PubMed   Google Scholar  

Kinsella KG, Phillips DR. Global aging: The challenge of success. 1st ed. Population Reference Bureau: Washington; 2005.

Nolte E, McKee M. Caring for people with chronic conditions: A health system perspective. 1 st ed. London: McGraw-Hill. Education. 2008;

Goodwin N, Curry N, Naylor C, Ross S, Duldig W. Managing people with long-term conditions. In: An inquiry into quality of general practice in England. The Kings Fund. 2010. http://www.kingsfund.org.uk/sites/files/kf/field/field_document/managing-people-long-term-conditions-gp-inquiry-research-paper-mar11.pdf . Accessed 17 Oct 2016.

Hackett ML, Yapa C, Parag V, Anderson CS. Frequency of depression after stroke. Stroke. 2005;36:1330–40.

Haines L, Wan KC, Lynn R, Barrett TG, Shield JP. Rising incidence of type 2 diabetes in children in the UK. Diabetes Care. 2007;30:1097–101.

Wolfe CD. The impact of stroke. Br Med Bull. 2000;56(2):275–86.

Choice Access Team. Understanding the benefits. In: Improving access to psychological therapies (IAPT) commissioning toolkit. Department of Health. 2008. https://www.uea.ac.uk/documents/246046/11991919/IAPT+Commissioning+Toolkit+2008+.pdf/cc6a4f24-dc6b-45d9-a631-ffdd075c6f0a . Accessed 17 Oct 2016.

Halpin DM, Miravitlles M. Chronic obstructive pulmonary disease: The disease and its burden to society. Prom Am Tharacic Soc. 2006;3(7):619–23.

Article   Google Scholar  

Wimo A, Winblad B, Aguero-Torres H, Von Strauss E. The magnitude of dementia occurrence in the world. 2003. Alzheimers Dis Assoc Disord. 2003;17(2):63–7.

Akehurst RL, Brazier JE, Mathers N, O’Keefe C, Kaltenthaler E, Morgan A, et al. Health-related quality of life and cost impact of irritable bowel syndrome in a UK primary care setting. PharmacoEconomics. 2002;20(7):455–62.

Nimnuan C, Hotopf M, Wessely S. Medically unexplained symptoms: An epidemiological study in seven specialities. J Psychosom Res. 2001;51(1):361–7.

Konnopka A, Schaefert R, Heinrich S, Kaugmann C, Luppa M, Herzog W, et al. Economics of medically unexplained symptoms: A systematic review of the literature. Psychother Psychosom. 2012;81(5):265–75.

Feinstein AR. The pre-therapeutic classification of co-morbidity in chronic disease. J Chronic Dis. 1970;23(7):455–68.

Fortin M, Bravo G, Hudon C, Vanasse A, Lapoint L. Prevalence of multi-morbidity among adults seen in family practice. Ann Fam Med. 2005;3(3):223–8.

Article   PubMed   PubMed Central   Google Scholar  

Boerma JT, Stansfield SK. Health statistics now: are we making the right investments? Lancet. 2007;369(9563):779–86.

Murray CJ. Towards good practice for health statistics: lessons from the Millennium Development Goal health indicators. Lancet. 2007;369(9564):862–73.

Alonso J, Ferrer M, Gandbeck B, Ware Jr JE, Aaronson NK, Mosconi P, et al. Health-related quality of life associated with chronic conditions in eight countries: results from the International Quality of Life Assessment (IQOLA) Project. Qual Life Res. 2004;13(2):282–98.

Ogden J. Health Psychology: A textbook. 5th ed. London: McGraw-Hill Education (UK; 2012.

Albrecht GL, Fitzpatrick R. A sociological perspective on health-related quality of life research. In: Advances in medical sociology, quality of life in health care. London: Jai Press; 1994. p. 1–21.

Bradley C. Importance of differentiating health status from quality of life. Lancet. 2001;357(9249):7–8.

WHOQoL Group. The development of the World Health Organisation quality of life assessment instrument (the WHOQOL). In: Quality of life assessment: International Perspectives. Berlin: Springer Berlin Heidelberg; 1994. p. 41–57.

Book   Google Scholar  

Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organisation for Research and Treatment of Cancer QLQ-C30: A quality of life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–76.

Fallowfield L. The quality of life: The missing measurement in health care. 1st ed. London: Souvenir Press; 1990.

Goldberg D, Williams P. General health questionnaire (GHQ). In: Swindon: nferNelson; 1988.

Hays RD. RAND-36 health status inventory. San Antonio: Psychological Corporation; 1998.

Hickey AM, Bury G, O’Boyle CA, Bradley F, O’Kelly FD, Shannon W. A new short form individual quality of life measure (SEIQoL-DW): Application in a cohort of individuals with HIV/AIDS. BMJ. 1996;313(7048):29.

Skevington SM, O’Connell KA. Measuring quality of life in HIV and AIDS: A review of the recent literature. Psychol Health. 2003;18(3):331–50.

Skevington SM, O’Connell KA. Can we identify the poorest quality of life? Assessing the importance of quality of life using the WHOQOL-100. Qual Life Res. 2004;13(1):23–34.

Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–70.

Furlong WJ, Feeny DH, Torrance GW, Barr RD. The Health Utilities Index (HUI®) system for assessing health-related quality of life in clinical studies. Ann Med. 2001;33(5):375–84.

Kaplan RM, Anderson JP, Ganiats TG. The quality of well-being scale: rationale for a single quality of life index. In: Quality of life assessment: Key issues in the 1990s. Amsterdam: Springer; 1993. p. 65–94.

Chapter   Google Scholar  

Ware Jr JE, Kosinski M, Keller SD. A 12-item short-form health survey: Construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.

American Psychiatric Association. Diagnostic and Statistical Manual, 5 th Edition (DSM-V). 5th ed. Washington: American Psychiatric Association; 2013.

Bener A, Al-Kazaz M, Ftouni D, Al-Harthy M, Dafeeah EE. Diagnostic overlaps of depressive, anxiety, stress and somatoform disorders in primary care. Asia Pac Psychiatry. 2013;5(1):E29–8.

Gilbert P. Depression: The evolution of powerlessness. 1st ed. Hove: Psychological Press; 2014.

Spielberger CD. Anxiety: Current trends in theory and research. 1st ed. Philadelphia: Elsevier; 2013.

Jackson-Koku G. Beck Depression Inventory. Occup Med. 2016;66(2):174–5.

Steer RA, Beck AT. Beck Anxiety Inventory. In: Zalaquett CP, Wood RJ, editors. Evaluating stress: A book of resources. Lanham: Scarecrow Education; 1997. p. 23–40.

Tennant R, Hiller L, Fishwick R, Platt S, Jospeh S, Weich S, et al. The Warwick-Edinburgh mental well-being scale (WEMWBS): development and UK validation. Health Qual Life Outcomes. 2007;5(1):1.

Moussavi S, Chatterji S, Verdes E, Tandon A, Patel V, Ustun B. Depression, chronic diseases, and decrements in health: Results from the World Health Surveys. Lancet. 2007;370(9590):851–8.

Sareen J, Jabobi F, Cox BJ, Belik SL, Clara I, Stein MB. Disability and poor quality of life associated with comorbid anxiety disorders and physical conditions. Arch Intern Med. 2006;166(19):2109–16.

Ganz PA, Desmond KA, Leedham B, Rowland JH, Meyerowtiz BE, Belin TR. Quality of life in long-term, disease-free survivors of breast cancer: A follow-up study. J Natl Cancer Inst. 2002;94(1):39–49.

Karakoyun-Celik O, Gorken I, Sahin S, Orcin E, Alanyali H, Kinay M. Depression and anxiety levels in woman under follow-up for breast cancer: a relationship to coping with cancer and quality of life. Med Oncol. 2010;27(1):108–13.

Margolis G, Goodman RL, Rubin A. Psychological effects on breast-conserving cancer treatment and mastectomy. Psychosomatic. 1990;31(1):33–9.

Spiegel D, Giese-Davis J. Depression and cancer: mechanisms and disease progression. Biol Psychiatry. 2003;54(3):269–82.

Watson M, Haviland JS, Greer S, Davidson J, Bliss JM. Influence of psychological response on survival in breast cancer: a population-based cohort study. Lancet. 1999;354(9187):1331–6.

Davies SJ, Jackson PR, Potokar J, Nutt DJ. Treatment of anxiety and depressive disorders in patients with cardiovascular disease. BMJ. 2004;328(7445):939–43.

Frasure-Smith N, Lesperance F, Juneau M, Talajic M, Bourassa MG. Gender, depression, and one-year prognosis after myocardial infarction. Psychosom Med. 1999;61(1):26–37.

Burns H. Health in Scotland 2007: Annual report of the chief medical officer. 1st ed. The Scottish Government: Edinburgh; 2009.

Sanders C, Egger M, Donovan J, Tallon D, Frankel S. Reporting on quality of life in randomised controlled trials: Bibliographic study. BMJ. 1998;317(7167):1191–4.

World Health Organisation. 2008–2013 action plan for the global strategy for the prevention and control of non-communicable disease: Prevent and control cardiovascular diseases, cancers, chronic respiratory diseases and diabetes. In: 2008–2013 Action Plan. World Health Organisation. 2009. http://apps.who.int/iris/bitstream/10665/44009/1/9789241597418_eng.pdf. Accessed 18 July 2016. Accessed 17 Oct 2016.

Cohen S, Herbert TB. Health Psychology: Psychological factors and physical disease from the perspective of human psychoneuroimmunology. Annu Rev Psychol. 1996;47(1):113–42.

Prince M, Patel V, Saxena S, Maj M, Maselko J, Phillips MR, et al. No health without mental health. Lancet. 2007;370(9590):859–77.

Galway K, Black A, Cantwell M, Cardwell CR, Mills M, Donnelly M. Psychosocial interventions to improve quality of life and emotional wellbeing for recently diagnosed cancer patients. Psycho-Oncology. 2013;22:253–4.

Happell B, Davies C, Scott D. Health behaviour interventions to improve physical health in individuals diagnosed with mental illness: A systematic review. Int J Ment Health Nurs. 2012;21(3):236–47.

Harkness E, Macdonald W, Valderas J, Coventry P, Gask L, Bower P. Identifying psychosocial interventions that improve both physical and mental health in patients with diabetes a systematic review and meta-analysis. Diabetes Care. 2010;33(4):926–30.

Hutchison AJ, Breckon JD. A review of telephone coaching services for people with long-term conditions. J Telemed Telecar. 2011;17(8):451–8.

Rehse B, Pukrop R. Effects of psychosocial interventions on quality of life in adult cancer patients: meta analysis of 37 published controlled outcome studies. Patient Educ Couns. 2003;50(2):179–86.

Simpson R, Booth J, Lawrence M, Byrne S, Mair F, Mercer S. Mindfulness based interventions in multiple sclerosis-a systematic review. BMC Neurol. 2014;14(1):165.

Stinson J, Wilson R, Gill N, Yamada J, Holt J. A systematic review of internet-based self-management interventions for youth with health conditions. J Pediatr Psychol. 2009;34(5):495–510.

Trautmann E, Lackschewitz H, Kröner-Herwig B. Psychological treatment of recurrent headache in children and adolescents–a meta-analysis. Cephalalgia. 2006;26(12):1411–26.

Van Beugen S, Ferwerda M, Hoeve D, Rovers MM, Spillekom-Van Koulil S, Van Middendorp H, et al. Internet-based cognitive behavioral therapy for patients with chronic somatic conditions: a meta-analytic review. J Med Internet Res. 2014;16(3):e88.

Sibbald B, Roland M. Understanding controlled trials. Why are randomised controlled trials important? BMJ. 1998;316(7126):201.

Hoffman BM, Papas RK, Charkoff DK, Kerns RD. Meta-analysis of psychological interventions for chronic low back pain. Health Psychol. 2007;26(1):1.

Rose MJ, Reilly JP, Pennie B, Bowen-Jones K, Stanley IM, Slade PD. Chronic low back pain rehabilitation programs: a study of the optimum duration of treatment and a comparison of group and individual therapy. Spine J. 1997;22(19):2246–51.

Ganaan R, Ciliska D, Thomas H. Expediting systematic reviews: Methods and implications of rapid reviews. Implement Sci. 2010;5(1):56.

Khangura S, Konnyu K, Cushman R, Grimshaw J, Mosher D. Evidence summaries: The evolution of a rapid review approach. Syst Rev. 2012;1(1):1.

Featherstone RM, Dryden DM, Foisy M, Guise JM, Mitchell MD, Paynter RA, et al. Advancing knowledge of rapid reviews: An analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst Rev. 2015;4(1):1.

Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions. 5th ed. Chichester: Wiley-Blackwell; 2008.

Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: providing the research evidence for public health nursing interventions. Worldviews Evid Based Nurs. 2004;1(3);176–84.

Braeken AP, Kempen GI, Eekers DB, Houben R, Gils GC, Ambergen T, et al. Psychosocial screening effects on health-related outcomes in patients receiving radiotherapy. A cluster randomised trial. Psychooncology. 2013;22(12):2736–46.

Jensen AM, Ramasamy A, Hall MW. Improving General Flexibility with a Mind-Body Approach: A Randomized, Controlled Trial Using Neuro Emotional Technique®. J Strength Cond Res. 2012;26(8):2103–12.

Lawler SP, Cameron LD. A randomized, controlled trial of massage therapy as a treatment for migraine. Ann Behav Med. 2006;32(1):50–9.

Mosher CE, Lipkus I, Sloane R, Snyder DC, Lobach DF, Demark-Wahnefried W. Long-term outcomes of the FRESH START trial: Exploring the role of self-efficacy in cancer survivors’ maintenance of dietary practice and physical activity. Psychooncology. 2013;22(4):876–85.

Pulgaron ER, Salamon KS, Patterson CA, Barakat LP. A problem-solving intervention for children with persistent asthma: A pilot of a randomized trial at a pediatric summer camp. J Asthma. 2010;47(9):1031–9.

Tabolli S, Pagliarello C, Sampogna F, Di Pietro C, Abeni D, GISED ICS. Evaluation of the impact of writing exercises and education interventions on quality of life in patients with psoriasis. Value Health. 2011;14(7):A509–10.

Yardley L, Joseph J, Michie S, Weal M, Wills G, Little P. Evaluation of a Web-based intervention providing tailored advice for self-management of minor respiratory symptoms: exploratory randomized controlled trial. J Med Internet Res. 2010;12(4):e66.

Baptist A, Ross JA, Yany Y, Song PX, Clark NM. A randomized controlled trial of a self-regulation intervention for older adults with asthma. J Am Geriatr Soc. 2013;61(5):747–53.

Blank MB, Hennessy M, Eisenberg MM. Increasing quality of life and reducing HIV burden: The PATH+ intervention. AIDS Behav. 2014;18(4):716–25.

Escobar JI, Gara MA, Diaz-Martinez AM, Interian A, Warman M, Allen LA, et al. Effectiveness of a time-limited cognitive behaviour therapy-type intervention among primary care patients with medically unexplained symptoms. Ann Fam Med. 2007;5(4):328–35.

Smeulders ES, Van Haastregt J, Ambergen T, Uszko-Lencer NH, Janssen-Boyce JJ, Gorgels AP, et al. Nurse-led self-management group programme for patients with congestive heart failure: randomized controlled trial. J Adv Nurs. 2010;66(7):1487–99.

Somers TJ, Blumenthal JA, Guilak F, Kraus VB, Schmitt DO, Babyak MA, et al. Pain coping skills training and lifestyle behavioral weight management in patients with knee osteoarthritis: a randomized controlled study. Pain. 2012;153(6):1199–209.

Van Der Meulen IC, May AM, Ros WJ, Oosterom M, Hordijk GJ, Koole R, et al. One-year effect of a nurse-led psychosocial intervention on depressive symptoms in patients with head and neck cancer: a randomized controlled trial. Oncologist. 2013;18(3):335–44.

Smeulders ES, Van Haastregt JC, Van Hoef EF, Van Eijk JT, Kempen GI. Evaluation of a self-management programme for congestive heart failure patients: Design of a randomised controlled trial. BMC Health Serv Res. 2006;6(1):91.

Brown AJ, Smith LT, Craighead LW. Appetite awareness as a mediator in an eating disorders prevention programme. Eat Disord. 2010;18(4):285–301.

Janevic MR, Janz NK, Kaciroti N, Dodge JA, Keteyian SJ, Mosca L, et al. Exercise self-regulation among older women participating in a heart disease-management intervention. J Women Aging. 2010;22(4):255–72.

Kucukarslan SN, Thomas S, Bazzi A, Virant-Young D. Using self-regulation theory to examine patient goals, barriers and facilitators for taking medicine. Patient. 2009;2(4):211–20.

Sabaté E. Adherence to long-term therapies: Evidence for action. Geneva: World Health Organisation; 2003.

Boot WR, Simons DJ, Stothart C, Stutts C. The pervasive problem with placebos in psychology: Why active control groups are not sufficient to rule out placebo effects. Perspect Psychol Sci. 2013;8(4):445–54.

Shively M, Kodiath M, Smith TL, Kelly A, Bone P, Fettely L, et al. Effect of behavioural management on quality of life in mild heart failure. Patient Educ Couns. 2005;58(1):27–34.

Jones SC, Iverson D, Burns P, Evers U, Caputi P, Morgan S. Asthma ageing: An end user’s perspective-the perception problems with the management of asthma in the elderly. Clin Exp Allergy. 2011;41(4):471–81.

Yu DS, Lee DT, Kwong AN, Thompson DR, Woo J. Living with chronic heart failure: a review of qualitative studies of older people. J Adv Nurs. 2008;61(5):474–83.

Rief W, Heitmüller AM, Reisberg K, Rüddel H. Why reassurance fails in patients with explained symptoms–an experimental investigation of remembered probabilities. PLoS Med. 2006;3(8):e269.

McKiernan A, Steggles S, Guerin S, Carr A. A controlled trial of group cognitive behaviour therapy for Irish breast cancer patients. J Psychosoc Oncol. 2010;28(2):143–56.

Download references

Acknowledgements

NA would like to thank Julie Murray and Dr. Allyson McCollam at NHS Borders, and Dr. Hannah Dale and Dr. Lloyd Wallace at NHS Education for Scotland, for their ongoing support and advice throughout the review.

The authors declare that they did not receive any financial support for the present study.

Availability of Data and Materials

All data generated or analysed during this study are included in this published article.

Author information

Authors and affiliations.

Public Health Department, NHS Borders, Melrose, TD6 9BD, UK

Niall Anderson

School of Medicine, University of St Andrews, St Andrews, KY16 9TF, UK

Niall Anderson & Gozde Ozakinci

You can also search for this author in PubMed   Google Scholar

Contributions

NA was involved in all processes involved in the rapid SR, including: design, research, development, search, extraction, collation, analyses and reporting formulation. GO provided supervision, advice, feedback, and contributions towards all processes from data extraction onwards. Both authors read and approved the final manuscript.

Corresponding author

Correspondence to Niall Anderson .

Ethics declarations

Authors’ information.

NA graduated with a BSc (Hons.) Psychology degree from the University of Dundee in 2014, before graduating with a MSc Health Psychology degree from the University of St Andrews in 2015. NA conducted the review as part of employment as a Trainee Health Psychologist in NHS Borders, with research being conducted in affiliation with the University of St Andrews. NA aims to achieve Chartership as a British Psychological Society and Health & Care Professionals Council registered Health Psychologist in June 2018.

GO is a Chartered Health Psychologist and is a Senior Lecturer in Health Psychology within the University of St Andrews School of Medicine Division of Population and Behavioural Health Sciences.

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Competing interests.

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Cite this article.

Anderson, N., Ozakinci, G. Effectiveness of psychological interventions to improve quality of life in people with long-term conditions: rapid systematic review of randomised controlled trials. BMC Psychol 6 , 11 (2018). https://doi.org/10.1186/s40359-018-0225-4

Download citation

Received : 04 October 2017

Accepted : 13 March 2018

Published : 27 March 2018

DOI : https://doi.org/10.1186/s40359-018-0225-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

BMC Psychology

ISSN: 2050-7283

psychological intervention literature

Taylor and Francis Online homepage

Work & Stress

Systematic literature review of psychological interventions for first responders.

Sample our Behavioral Sciences journals, sign in here to start your access, latest two full volumes FREE to you for 14 days

We conducted a systematic review of the empirical literature examining the effectiveness of psychological interventions for post-traumatic symptomatology in police, firefighters, and paramedic personnel. The review process was guided by the PRISMA statement (Moher et al. [2009]. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine , 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097) and Institute of Medicine’s Standards for Systematic Reviews (IOM [2011]. Finding what works in health care: Standards for systematic reviews . nihlibrary.nih.gov/sites/default/files/Finding_What_Works_in_Health_Care_StandardsforSystematic_Reviews_IOM_2011.pdf ). An inter-disciplinary, multi-national research team with expertise in mental health trauma and occupational stress in high risk professions was engaged at each stage of the review. Two team members rated each study in terms of quality and contribution to the research question. Twenty-one studies were identified: 9 case studies, 2 single-group studies, 8 randomised controlled trials, and 2 studies examining work leave. Most of the studies were limited by small sample sizes and absence of active control conditions. Research limitations reduce the ability to draw definitive best practices recommendations; however, the increase in randomised controlled trials provides encouraging signs that trauma-focused psychotherapies can be effective for first responders.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Log in via your institution, log in to taylor & francis online, restore content access.

Related Research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations. Articles with the Crossref icon will open in a new tab.

Your download is now in progress and you may close this window

Login or register to access this feature

Register now or learn more

People also looked at

Systematic review article, effectiveness of mobile app-based psychological interventions for college students: a systematic review of the literature.

psychological intervention literature

Serious mental health disorders are increasing among college students and university counseling services are often overburdened. Mobile applications for mental health have been growing exponentially in the last decade and they are emerging in university settings as a promising tool to promote and intervene in college students' mental health. Additionally, considering the recent covid-19 pandemic, mHealth interventions, due to its nature and possibilities, may play an important role in these institutions. Our main objectives are to explore mhealth interventions in universities, regarding its conceptual framework, acceptability and efficacy outcomes and understand its impact and contributions to address treatment delivery and psychological difficulties resulting from covid-19 pandemic. The literature search was conducted in scientific databases, namely, Web of Science, Pubmed, and Scopus. A search in app stores was not conducted, thus regarding commercially available apps, only those found in our database search were included in our review. We selected studies with mobile applications addressing psychological interventions for college students. A total of 2,158 participants were included in the 8 selected studies and most interventions were delivered through mobile apps only and based in cognitive behavioral therapy. Results suggested that college students accept and adhere to these interventions and preliminary evidence of efficacy was demonstrated in different disorders, such as stress, anxiety, depression and risky behaviors such as alcohol and tobacco abuse and sexual knowledge. We conclude that universities, particularly college counseling services, may benefit from mhealth interventions, not only to address college students' mental health but to decrease some of its difficulties related to lack of human resources. Specifically in covid-19 pandemic context, these interventions may contribute significantly by promoting and delivering psychological interventions at a safe distance.

Introduction

Over the last decade numerous mental health mobile applications have been developed and made available for users ( Bakker et al., 2016 ). Smartphones demonstrate numerous advantages such as great computing capacity, mobility, and more rapid and efficient access to information by using mobile applications ( Donker et al., 2013 ). The enthusiasm of smartphones for healthcare initiatives led to the emergence of a novel field called mHealth ( Ben-Zeev et al., 2014 ) defined as the use of mobile technologies to deliver or support psychological or mental health interventions and includes mobile devices such as smartphones, tablets, Personal Digital Assistants, and wearable devices ( Clough and Casey, 2015b ; Alyami et al., 2017 ). In clinical settings, mHealth may enhance face-to-face treatments, increase patient engagement in therapy sessions and adherence to therapy principles; provide better use of clinician time and resources and improve treatment outcome and risk of relapse ( Clough and Casey, 2015b ). Several studies have shown that mental health apps and cognitive behavioral therapy (CBT)-based apps are efficacious ( Rathbone et al., 2017 ; Linardon et al., 2019 ). However, despite clinical potential, interest and early supporting evidence, one factor that seems to limit mental health apps is low engagement or poor adherence to the intervention ( Torous et al., 2018 ).

One of the areas were mental health apps can have a significant impact is in universities. College years are a sensitive period to the onset of several mental health disorders ( Kessler et al., 2007 ) and many studies have reported a significant rise in serious mental health illness among college students ( Hunt and Eisenberg, 2010 ; Storrie et al., 2010 ; Auerbach et al., 2018 ). Major Depressive Disorder (MDD) and Generalized Anxiety Disorder (GAD) were identified as the most common disorders found in college students ( Auerbach et al., 2018 ). University counseling services constitute a valuable resource to support college student mental health and wellness ( Spooner, 2000 ) and a challenge that seems to be common across several counseling services is the growing student demand for these services and the limited resources to face these demands ( Johnson and Kalkbrenner, 2017 ; Shaw et al., 2017 ; Auerbach et al., 2018 ; Lee and Jung, 2018 ). College students are also large consumers of technology and communicate frequently online ( Shaw et al., 2017 ). A study by Wilansky et al. (2016) referred that mobile applications may increase youth adherence to Cognitive Behavioral Therapy (CBT) and improve treatment outcomes. Research suggests that mHealth is already being used to increase students' awareness and to deliver health-related interventions with increasing popularity; preliminary findings indicate that students are open and willing to use these interventions ( Johnson and Kalkbrenner, 2017 ).

Mobile technologies for mental health assume an important role considering our current reality of pandemics resulting from covid-19 infectious disease. Covid-19 is an infectious disease cause by a coronavirus that rapidly expanded worldwide, and some of the protective measures include physical distancing, wearing a mask, avoiding crowds and close contact, and regularly cleaning your hands ( World Health Organization, 2020 ). College students, alongside with children and health workers, are one of the most exposed groups to develop post-traumatic stress disorder, anxiety, depression and other symptoms of distress ( Saladino et al., 2020 ). Studies conducted during covid-19 pandemic in China concluded that almost half of Chinese college students that participated in the study experienced anxiety symptoms ( Fu et al., 2021 ) and are more likely to suffer from stress, anxiety and depression than the general population ( Li et al., 2020 ). Several studies highlight the need to monitor students' mental health during the pandemic and the delivery of timely and appropriate interventions ( Cao et al., 2020 ; Fu et al., 2021 ) such as the importance of technological devices or digital interventions ( Saladino et al., 2020 ). Covid-19 brought several challenges to mental health services delivery, thus many therapists rapidly adhered to telehealth to replace in-person contact ( Taylor et al., 2020 ). The same authors state that this disease presents an imperative for mental health services to make digital mental health interventions available in routine care and not only in response to covid-19 crisis.

Previous systematic reviews with college students and mobile interventions often explore a wide range of mHealth interventions and technology (e.g., Johnson and Kalkbrenner, 2017 ). Our review will focus on (1) mental health mobile applications that include a psychological intervention targeting a mental disorder, (2) college students, and (3) randomized controlled trials and acceptability and feasibility studies. We aim to explore how mobile apps are being developed to address college students' mental health in universities, if they accept and adhere to these interventions and if these interventions demonstrate efficacy. A search will be made for peer-reviewed articles of mental health mobile apps in scientific databases. The present review will not conduct a search in app stores mainly because acceptability and efficacy outcomes are not usually reported in app stores and because it would demand a different type of search strategy. Thus, in the current review we aim to review all published literature, in scientific databases, on psychological interventions using mobile applications, in the last 12 years, for college students. Our main objective is to review efficacy outcomes, through randomized controlled trials, of mobile app-based psychological interventions compared to traditional therapy or a waiting list control group in reducing psychological symptomatology among college students. Additionally, we intend to explore how mobile interventions are being accepted by college students and which conceptual frameworks are being used to develop these interventions. Considering the recent context of covid-19 pandemics, we aim to reflect on the impact and contributions of mHealth interventions for universities and college students.

We used the search method of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) ( Moher et al., 2009 ).

Eligibility Criteria and Information Sources

Inclusion criteria considered (1) target population: college students; (2) types of intervention: psychological interventions delivered through mobile applications (self-guided); mobile applications combined with web-based interventions or mobile applications combined with face-to-face treatments; (3) Primary outcome measures that target specific psychological disorders or symptomatology (i.e., anxiety, depression, social anxiety, stress, PTSD, alcohol abuse); (4) clear report of the psychological intervention, specifying theoretical basis or treatment model and therapeutic techniques; (5) Types of studies: randomized controlled trials (RTC) or quasi-experimental designs that clearly report efficacy outcomes and feasibility and acceptability studies since they contribute with valuable information about conceptual framework and some provide preliminary effectiveness results; (6) written in English; (7) published in the selected scientific databases. Exclusion criteria consisted in (1) studies with young adults (not students); (2) mobile interventions based on text messages; (3a) mobile interventions targeting physical or medical conditions (e.g., diabetes, physical activity, nutrition, weight control etc.); (3b) studies about mobile learning apps (e.g., anatomy); (3c) studies about smartphone addiction; (4) internet and computerized based interventions; (5) study protocols.

Our main objective is to review conceptual framework, acceptability, and efficacy outcomes of mobile app interventions addressing mental health for college students. A search of mobile apps commercially available in the app store was not conducted in this review since, although important, demands a different type of search and selection process, and often don't report acceptability and efficacy results (in the app store). Thus, we considered that it would be more suited to do a review, with this group of apps, separately. A narrative approach was used for extraction and synthesis of the data. Studies were identified through three major electronic databases, namely, Web of Science, Pubmed, and Scopus. An update literature search was performed in January 2021 using the same information sources.

Search and Study Selection Process

The following search keywords were considered “mobile interventions,” “smartphones,” “mobile application,” “mHealth,” “mobile technology,” “college students,” “students,” “university,” “campus.” Two authors independently conducted a thorough search in the three major scientific databases with the mentioned keywords, using primarily the combination “mobile interventions” AND “college students” with year filter between 2008 and 2019. A search update was performed in January 2021 with the same study selection process. In a first instance, studies including keywords in titles and/or abstracts were selected for further thorough review. After identifying eligible studies, duplicates were removed, and full papers were examined regarding eligibility criteria. A list of studies was produced by each author. Afterwards, both authors discussed their list of included studies, and by agreement, a final list of studies was produced.

Data Extraction

Data extraction was performed by two independent researchers and included year of publication, demographic characteristics of participants, study design (RCT, quasi-experimental studies, single-arm pre-test post-test), study participants and interventions (i.e., population, conditions, sample size, outcome measures, mobile app characteristics, theoretical basis, and intervention modality), main results and findings.

Assessment of Methodological Quality

The present review resorted to critical appraisal tools from the Joanna Briggs Institute for randomized controlled trials and quasi experimental studies (non-randomized experimental studies). The Checklist for Randomized Controlled Trials [ The Joanna Briggs Institute (JBI), 2017a ] was utilized to assess the methodological quality of the included RCTs and the Checklist for Quasi-Experimental Studies (non-randomized experimental studies) [ The Joanna Briggs Institute (JBI), 2017b ] to assess methodological quality of quasi-experimental studies and studies with a one group pre-test post-test design. Each study was assessed using JBI checklists for RCT or quasi-experimental studies.

Study Selection

As we can see in Figure 1 our search identified 957 published articles. Afterwards, we removed 23 duplicates and a review of title and abstracts excluded 904 articles. A total of 30 full-text articles were assessed for eligibility, where 11 were excluded due to motives of being a study protocol, thus not presenting feasibility or efficacy outcomes; lack of a psychological intervention or a psychological disorder; being web-based intervention or having no access to article full text. A total of 19 studies were included and examined in accordance with inclusion criteria.

www.frontiersin.org

Figure 1 . PRISMA flow diagram. From: Moher et al. (2009) .

Demographic Characteristics

A total of 3,399 college students were included in the selected studies ( n = 19) for this systematic review. Eleven studies included college students with self-reported psychological symptomatology (i.e., elevated stress, generalized anxiety disorder (GAD), PTSD), two studies included first-year college students and the remaining studies included non-treatment seeking college students ( n = 6). Most studies occurred in the USA ( n = 12), others occurred in Germany ( n = 1), Sweden ( n = 1), Canada ( n = 1), United Kingdom (UK) ( n = 2), Australia ( n = 1), and Iran ( n = 1).

Intervention Characteristics

Mobile intervention apps for college students target anxiety ( n = 7), depression ( n = 7), stress ( n = 5), alcohol consumption and risky drinking ( n = 4), smoking ( n = 1), and sexual behaviors ( n = 1), Post-traumatic stress disorder (PTSD) ( n = 1). Table 1 resumes all further interventions characteristics.

www.frontiersin.org

Table 1 . Mobile interventions characteristics.

We considered that most studies, with self-guided apps, focus on prevention ( n = 15) and the studies that included human support (therapists and coaches) and a TAU group were more focused on a treatment approach ( n = 4). However, many studies with self-guided apps, included students with elevated psychological symptomatology (i.e., elevated stress, diagnosed PTSD, or GAD), and it isn't always clear the nature of their intervention.

Intervention modality varied between a combination of internet and mobile app intervention ( n = 2) and mobile app intervention only ( n = 17), from these 17 studies, two apps were combined with a wearable band to permit passive data collection. When combining mobile apps with internet interventions, the mobile app functioned mostly as a tool offering support for homework assignment or working as a diary app by enabling monitoring of mood fluctuations or stress levels [e.g., Harrer et al. (2018) ]. Human support was considered in 4 mobile apps (Lantern; TAO; StudiCare Stress; Mind the Moment), two mobile interventions included therapists and two included a coach, StudiCare Stress app included a trained master's student in Psychology (named an eCoach) and Lantern app included a coach with various educational backgrounds. Human support varied from weekly 10–12 min brief videoconferences, to 2 face to face sessions and online sessions only.

Regarding conceptual framework most researchers used CBT intervention or CBT third wave techniques to conceptualize these apps ( n = 17). Most CBT apps include mindfulness exercises ( n = 11), some are solely based on mindfulness ( n = 4) or acceptance and commitment therapy (ACT) ( n = 2). One mobile app is focused on CBT and a biofeedback intervention (BioBase app). Some used CBT intervention as a part of a larger program such as GET.ON Stress, a stress management program, adapted to college students; or BASICS, an alcohol intervention program for college students. In some cases, CBT was combined with other psychological models such as Lazarus Transactional Model of Stress (GET.ON Stress program) or the Unified Theory of Use and Acceptance of Technology (UTAUT). The StudiCare Stress app also included an adherence-focused guidance concept according to the human accountability model. Only two studies did not resort to CBT, the SmarTrek app that used motivational interviewing and the SEX101 that used two psychological models, the Theory of Reasoned Action (TRA; Fishbein and Ajzen, 1975 ) and the Transtheoretical Model (TTM) of behavior Change ( Prochaska and DiClemente, 1984 ). Additionally, SmartTreak and MtM added an Ecological Momentary Intervention (EMI) and Witkiewitz et al. (2014) , BioBase app and ACT daily included an Ecological Momentary Assessment (EMA).

As for specific techniques more than half of the mobile apps include mindfulness exercises; other included psychoeducation or general information about the target disorder; include data collection self-monitoring; exposure; systematic desensitization and relaxation exercises. Other features refer to quizzes and interactive games; virtual coach; passive sensing through sensorband; all apps for risky drinking and excessive smoking included personalized feedback on drinking patterns and motives for drinking, feedback includes information about smoking and “urge-surfing” or strategies to increase student's emotional awareness. All apps were designed to provide education, collect data, monitor/track behavior, some provide personalized feedback or guidance in CBT exercises (in some cases homework assignments).

Few studies gave information regarding privacy and security. For example, Benton et al. (2016) referred that TAO security and privacy included authentication, password protection, and encryption of databases and Lee and Jung (2018) stated that data was collected and stored on secure systems and accessed through computers with password protection and encryption.

Methodological Quality

Tables 2 , 3 resumes the methodological characteristics of the included studies. Eleven studies are randomized controlled trials (RCT) ( Witkiewitz et al., 2014 ; Gajecki et al., 2017 ; Harrer et al., 2018 ; Lee and Jung, 2018 ; Fish and Saul, 2019 ; Huberty et al., 2019 ; Bruehlman-Senecal et al., 2020 ; Flett et al., 2020 ; McCloud et al., 2020 ; Newman et al., 2020 ; Ponzo et al., 2020 ) and two studies are considered quasi-experimental trials ( Benton et al., 2016 ; Borjalilu et al., 2019 ). Four studies considered a single-arm pre-test-post-test study design ( Jackson et al., 2016 ; Leonard et al., 2017 ; Haeger et al., 2020 ; Lattie et al., 2020 ; Reyes et al., 2020 ) and one study included two groups through an iterative process ( Kazemi et al., 2018 ).

www.frontiersin.org

Table 2 . Methodological characteristics.

www.frontiersin.org

Table 3 . JBI Checklist for randomized controlled trials.

Eleven of the included studies are RCTs and the total sample size ranges from 72 to 330 college student participants; the overall duration of the intervention range from 14 days to 3 months and when we consider follow-ups, the longest trial lasted for 9 months. Most RCTs included as a control group a waiting list control trial ( n = 8). Following JBI critical appraisal tool, we consider that all RCTs reported that participants were randomly assigned to treatment groups, 9 out of 11 studies provided detailed description of the randomization procedure and two studies merely stated that the participants were randomly assigned. As for allocation concealment, three studies provided information about allocation concealment. For example, Harrer et al. (2018) stated that the randomization process was performed by a researcher not involved in the study, and although they weren't able to blind participants to study conditions, during the randomization process, they were able to conceal the allocation from participants, researchers, and e-coches. Ten studies provided information and reported similar groups at baseline. As for blinding participants, or those delivering treatment and even outcome assessors to treatment conditions may be difficult and even unachievable in this type of studies; several studies reported this issue, pointing to the inability to blind their participants to treatment conditions. There were incomplete follow-ups; however they were generally adequately described and analyzed. Six RCTs provided detailed information about intention-to-treat analyses (ITT); the remaining studies excluded participants, lost to follow-up, from analysis. All studies used primary outcome measures with good validity and reliability. The large majority of RCTs also included quantitative and/or qualitative self-report measures to evaluate usability, acceptability, user satisfaction, or app adherence.

The studies by Benton et al. (2016) and Borjalilu et al. (2019) were considered as quasi-experimental studies. The first study included a large sample size ( n = 1,241) with overall duration of the intervention of 7 weeks. They included a wait-list treatment as usual control group and the intervention group received the intervention of study. The primary outcome measure was adequately validated and provided multiple measurements along the intervention as well as pre and post assessment. Differences between groups in terms of follow-up were adequately described and analyzed. This study presented many missing data and the linear mixed-effects models was utilized to estimate parameters for missing values. As for Borjalilu et al. (2019) , they conducted a study with three conditions and 68 college students, who were randomly assigned into the three groups, but no further detailed information was given about the randomization process. There were pre- a post-assessments and follow up was complete. Outcomes were measured in a reliable way and participants, from both groups, were assessed in the same way.

In this review there is a significant number of a single group pre-test-post-test design studies that aimed to evaluate acceptability and feasibility; only one study ( Jackson et al., 2016 ) aimed to evaluate efficacy with this design. Sample sizes were similar between studies, ranging from n = 10 to n = 23, with overall duration (intervention) of 3–4 weeks. Adequate and validated main outcome measures were used. The SEX101 ( Jackson et al., 2016 ) had a larger sample size compared to the previous studies and a follow-up assessment of 3 months after intervention completion. However, the overall duration of the intervention was very small (pre-test and intervention had to be complete in 1 week and it takes 40 min to complete) and some outcome measures were developed by the researchers with few information regarding reliability.

Intervention Outcomes and Effect Sizes

A study conducted by Newman et al. (2020) assessed the efficacy of Lantern, a self-help mobile app to treat generalized anxiety disorder. Study results demonstrated a significant reduction on the DASS stress scores ( d = 0.408) and greater probability of remission from GAD ( d = 0.114). Lantern revealed moderate effects in reducing anxiety, stress, and depression. BioBase is a biofeedback self-guided mobile app combined with wearable device (BioBeam), to treat anxiety in college students. Ponzo et al. (2020) conducted a RCT to assess BioBase efficacy and results indicated that a 4-week intervention significantly reduced anxiety ( d = 0.67), depression (d = 0.99), and increased perceived well-being ( d = 0.65) demonstrating moderate to large effects. Sustained large effects at 2-week follow-up was found for anxiety ( d = 0.81) and perceived well-being ( d = 1.16).

McCloud et al. (2020) conducted a RCT to assess efficacy of Feel Stress Free app for the treatment of depression and anxiety symptoms. Results showed that there was a significant reduction of depression symptoms at week 4 ( d = 0.27) and week 6 ( d = 0.39), and significant reduction of anxiety symptoms at week 4 ( d = 0.58). Overall effect sizes ranged from small to moderate.

Bruehlman-Senecal et al. (2020) studied Nod, a mobile app designed to reduce loneliness during the transition to college. Their RCT results indicated significant condition-by-baseline loneliness interaction to predict week-4 depression (Np 2 = 0.02) and sleep quality (Np 2 = 0.04), suggesting that Nod buffered participants with higher baseline loneliness against heightened midquarter depression and poor sleep quality. Calm, is a mindfulness-based app, and its efficacy was tested among students with elevated stress. The study results of Huberty et al. (2019) found significant differences among conditions in all outcomes, namely, significant reduction in perceived stress ( d = 1.24), significant improvements in mindfulness ( d = 1.11), and self-compassion ( d = 0.84).

Harrer et al. (2018) conducted a randomized controlled trial to evaluate the efficacy of Studicare Stress, a stress management intervention app for college students. Their results indicated significant effects of the intervention compared with the waitlist control group for stress at post-test ( d = 0.69) and at 3-month follow-up, other secondary outcome measures also yielded significant effects such as anxiety ( d = 0.76), depression ( d = 0.63), college related productivity ( d = 0.33), and academic work impairment ( d = 0.34). Thus, Studicare Stress revealed moderate to large intergroup effects for the reduction of perceived stress and other health and college related outcomes.

Lee and Jung (2018) conducted a pilot study to evaluate efficacy of DeStressify, a mindfulness-based app on stress, anxiety, depressive symptomatology, sleep behavior, and other variables. Results indicated that when using the app during 4 weeks, students in the experimental group at post-test reported less trait anxiety ( N p 2 = 0.040); an improve in several quality of life subscales, such as general health, that significantly differed between treatment condition in post-intervention scores ( N p 2 = 0.07). A significant difference was also found in energy or fatigue subscale between treatment conditions ( N p 2 = 0.05). An interaction effect was found in the emotional well-being subscale ( N p 2 = 0.05). The author interpreted the partial eta squared values of 0.0099, 0.0588, and 0.1379 as small, medium, and large effect, respectively, following suggestions by Cohen ( Field, 2009 ). This indicates that we can verify small (trait anxiety) to medium effects for general health, energy or fatigue and emotional well-being.

Telecoach app ( Gajecki et al., 2017 ) was evaluated using a 3-arm randomized controlled trial and results demonstrated that the proportion of students with excessive alcohol consumption declined in both intervention and wait list control group compared to controls at first and second follow-ups. Secondary analysis showed reductions for the intervention group in quantity of drinking at first follow up and in frequency of drinking at both follow-ups. Across both follow-ups the odds ratios for not having excessive weekly alcohol consumption in the intervention group (1.95) was almost twice as high as for controls (1.00). Secondary analysis by gender showed that the odds ratio for not having excessive alcohol consumption among men in the intervention group compared to male controls was higher (2.68) than women in the intervention group (1.71) compared to women controls.

Witkiewitz et al. (2014) conducted a 3-arm randomized controlled trial to evaluate a mobile feedback intervention for heavy-episodic drinking (HED) and smoking among college students, and they concluded that at 1-month follow-up there were significant reduction in number of cigarettes per smoking day in both the mobile intervention ( d = 0.55) and mobile assessment conditions ( d = 0.45) with moderate effects. No significant results were observed on HED or concurrent smoking and drinking. As for Benton et al. (2016) quasi-experimental study, the intervention group showed improvements across time significantly greater than treatment as usual participants, for all primary outcomes except Life Functioning (LF) subscale. The size of these effects ranged from small ( d = 0.16) for LF, Global Mental Health and Well-Being (d = 0.20) to medium for Anxiety ( d = 0.31).

Usability, Acceptability, and Feasibility Outcomes

The large majority of the included studies evaluated acceptability and students' satisfaction with the intervention. From the 19 studies, eight studies explored adherence/satisfaction and six used adequately valid scales or methods to assess usability or satisfaction with app use. Some studies also used metrics obtained through the mobile app ( n = 2). Most studies, created their own items to assess satisfaction with the intervention. Overall, we could observe good retention rates across studies, however as Gajecki et al. (2017) specifically noted in there study, there is a possibility that their fairly high retention levels could result from the desire of some participants to win an iPad (reward to participate in the study) with no actual intention to use the app. Out of the 19 studies, 10 gave rewards to their participants.

All studies that evaluated satisfaction reported moderate to high client satisfaction with the intervention. The MtM app ( Leonard et al., 2017 ) demonstrated that 60% of the participants reported “mostly” or “very” satisfied with the sensorband and 50% with the mobile app. Also, 93.9% of the participants were very satisfied or satisfied with the intervention program of SEX101 app ( Jackson et al., 2016 ). However, this particular study produced large attrition rates (50%) and as the authors of this study noted information regarding app components that need to be improved, added or removed should be collected. In the Witkiewitz et al. (2014) EMA app, over 65% of the participants reported an increase in awareness of their drinking and/or smoking and 60% stated that they would recommend this study to a friend because it provided greater awareness and they could help a friend reduce their drinking and/or smoking. Kazemi et al. (2018) demonstrated good usability of SmartTrek and the best feature reported by students was “Games” and the most useful features was “know your BAC” and “My strategies” that monitored alcohol intake, created behavioral change plans and reminded them of their goals. None of the studies, that provided human support (therapists), explored acceptability and satisfaction of the therapist with the intervention.

Implications and Contributions of mHealth Interventions for College Students in Covid-19 Context

Covid-19 infectious disease emerged in China and rapidly expanded around the globe, leading to an unexpected pandemic, which completely changed our daily lives and significantly limited physical and social contact with significant repercussions to our physical and mental health. Specifically in college students that live in a constant and thriving social interaction, covid-19 pandemic had a strong negative impact on mental health and may have contributed to the increase of several preexisting barriers and limitations to college counseling services. Considering these restraints, mHealth interventions may play an important role in a pandemic context due to its ubiquitous, remote and innovative functionalities that may facilitate access to evidence base treatments for mental health and also, its provider and facilitator (therapist).

Taking into consideration the included studies and their characteristics, acceptability, satisfaction and efficacy outcomes, we may determine that these interventions can significantly contribute in several important aspects related to college students' mental health. To our understanding, mobile app technologies may significantly contribute to promote mental health in college students targeting several specific disorders, such as anxiety, stress, depression, smoking, and alcohol abuse. It is also attainable to support students with coping strategies for elevated stress, anxiety, smoking, and alcohol abuse. Through mobile technologies, therapists may monitor and keep track of their patients' symptomatology and well-being, check homework assignments, and contact their patients' regularly through chat or messages, remotely. Overall, mobile technologies provide spontaneous and remote access to app content whenever we want, particularly in the comfort of our home. It helps us maintain physical distance from mental health professionals and counseling services without interrupting treatment.

Summary of Evidence

Our search for studies addressing mobile health apps for college students in university settings gathered 19 studies with different conceptual frameworks and study designs. In this review we could verify an increase in studies using mobile interventions for college students over the years, particularly in the last year, which may indicate an increasing trend in mobile use for the delivery of health interventions for college students. The large majority of studies are being developed in North America and Europe.

Regarding target disorders we can verify that most apps target anxiety, depression and stress, others target risky or excessive drinking, PTSD and sexual behaviors. Overall, mobile interventions showed promising results to reduce psychological symptomatology associated with stress, depression, anxiety and general student's mental health. As for drinking, smoking, and sexual behaviors, the included apps seemed to reduce excessive drinking and smoking and increase contraceptive use and knowledge but not the intention to reduce sexual risk behaviors or actual risk reduction. Most of the mobile interventions showed medium to large effect sizes for the main variables the app was designed to intervene, which may indicate that these interventions are well conceptualized and grounded according to the best available empirical evidence. Some of the included studies aimed to evaluate acceptability and feasibility and overall, these apps demonstrate good acceptability and feasibility among college students, supporting the hypothesis that students may accept and adhere to these interventions.

When we explore conceptual frameworks of these mobile apps we verify that many studies adopted CBT as the main intervention, particularly Mindfulness exercises. Effectively, CBT is well-established and particularly known as an effective treatment for several mental health disorders, and have demonstrated its efficacy when delivered through apps ( Rathbone et al., 2017 ). In some studies the intervention was complemented with psychological models, which have been shown to increase intervention efficacy ( Webb et al., 2010 ). Aside from psychological models/theories for behavioral change, one study incorporated a technological model, namely the Unified Theory of Use and Acceptance of Technology (UTUAT). There seems to be a strong application of psychological models and intervention techniques, indicating that there is a concern in adequately conceptualizing these interventions following evidence base principles. However, considering that we are studying mobile health interventions with significant emphasis in technology, very few studies incorporated technological models. Also, security and privacy features are also rarely mentioned and increasingly relevant in this type of interventions, best practices should be known and shared, reflecting in a mobile app quality indicator.

Regarding therapist role in mobile interventions, only 4 studies incorporated human support, two studies included therapists and two studies included a trained psychology student. From the mentioned studies, one used the human accountability model to inform this support. We consider that even though most of these apps intend to reduce therapist time and subsequently reduce therapist caseload and overburdened, this process may be optimized and better conceptualized using human support models. Moreover, evidence shows that app based interventions with therapist support has shown to produce larger effects ( Linardon et al., 2019 ).

As for methodological quality of the included studies, most studies aimed to evaluate efficacy and resorted to a randomized controlled trial, which is natural since RCTs are known as the golden standard to evaluate efficacy. All trials randomly assigned their participants to treatment conditions; however the number of studies that performed randomization concealment and blinding was almost non-existent. This reflects the difficulty of concealment and blinding in these type of studies and the limitation of the RCT study design when assessing efficacy in this type of interventions. Most studies also use a waiting list control group; given that many studies included students with elevated psychological symptomatology (that have to wait weeks/months to get access to the intervention) and the difficulty of blinding participants with this type of comparator we wonder if this is the best control group to use in this studies. Other research designs are also being explored in these studies and should be considered, so we can obtain efficacy results timelier and reliably ( Clough and Casey, 2015a ). Many studies adopted a pre-test post-test study design in order to evaluate acceptability and feasibility, even though this research design is considered a weak experimental study design, we consider that for the purpose and objectives of the studies this design was well-applied. Good overall retention rates may indicate treatment feasibility and acceptability. However, most studies were of short duration, with small samples and in controlled settings, with the addition of significant rewards. Additionally, many outcome measures were self-reported and not always congruent with app adherence rates. User metrics (e.g., how many times a participant accessed the app) provided by mobile apps may contribute to more accurate indicators of use and adherence to the intervention. Also, qualitative studies exploring perceived usefulness and user experience with the app intervention may also contribute to understand and overcome some barriers of adherence and engagement. Rewards are sometimes our best option to find participants, however when we are studying acceptability and adherence to these interventions, rewards may produce biased results. Recent studies opted to reward outcome measures completion, rather than app use.

A final question that emerged while exploring the studies is associated with the limited visual content of the apps included in the studies. Few studies included images/visual content of the mobile apps; some studies reported how they developed the app but provided little information about app design. A study by Torous et al. (2018) concluded that most mhealth apps suffer from low engagement and adherence and this may be, along with other issues, due to poor usability and because most apps are not user-friendly. It is important that researchers provide more frequently studies regarding user's needs and report multidisciplinary teams when building (native) apps, since this area often needs involvement of psychologists, software engineers, and designers/interaction designers. Also these tools, in clinical settings (e.g., counseling services), should be designed and optimized regarding all end users: students and therapists. Therapists' point of view and evaluation was often forgotten in the included studies that involved therapists.

Mobile apps may be customized and designed under practically unlimited possibilities. They can be developed to promote, prevent or intervene in a specific mental health disorder; to promote well-being and to deliver treatment under different levels of therapist support in different mental health services. Thus, they can be implemented and tailored according to specific needs. It is important to continue studying these interventions using user-centered designs and rigorous efficacy and effectiveness studies. We consider that universities, including college counseling services, may benefit from mhealth interventions, not only to address college student mental health but to decrease some of its difficulties related to few human resources. In a context of quarantine and confinement at home, where physical and social distance is imperative, these interventions assume special importance. They facilitate mental health promotion and support therapist and patient contact at a safe distance, avoiding treatment interruption.

Limitations

The current review presents a major limitation since we limited our search scope to the mentioned databases. Registered clinical trials and commercially available apps in app stores were not included, thus we may have missed already developed or apps that are being currently studied for college students. We may have failed to identify studies with relevant information regarding the application of mHealth intervention in college settings when we didn't consider “young adults,” since it may not include college students or occur in college settings.

The current systematic review shows that mobile apps for mental health intervention in college students exists and demonstrates good acceptability and feasibility. They also demonstrate efficacy among students. Overall we may conclude that mHealth interventions may turn out to be a great resource and tool to implement in counseling services, offering therapists and students many advantages. Particularly in the current pandemic context, these interventions demonstrate innumerous possibilities and promising solution to address college students' mental health and overcome many barriers associated with treatment access.

Future studies addressing mobile apps in college students, should invest in user-centered design studies so we can better understand what students and therapists (also attending university counseling services workflow) value more in a mobile based psychological intervention, to better adapt and tailor the intervention to user's needs. Effectively, acceptability and feasibility results among therapists are lacking in studies that use mobile intervention with therapist support. Future investigations should also explore diversity when developing and studying future apps, examining the applicability and efficacy of other theories/models. Also, we consider that studies should describe the development process of the mobile application (e.g., by including visual content) so we can better understand what is actually being evaluated and how it may impact efficacy results, in terms of usability and design. Lastly, students are large consumers of technology and so it may be important to invest more in these interventions, doing larger studies with more students, with superior methodological quality and avoiding large monetary rewards.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

CO searched for studies to include in the systematic review and wrote sections of the manuscript. AP and PV revised the manuscript and contributed to the conception of the study. CN, JG, and BA contributed to organize data extraction and the search of studies in the scientific databases. All authors contributed to the manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Alyami, M., Giri, B., Alyami, H., and Sundram, F. (2017). Social anxiety apps: A systematic review and assessment of app descriptors across mobile store platforms. Evid. Based Ment. Health 20, 65–70. doi: 10.1136/eb-2017-102664

PubMed Abstract | CrossRef Full Text | Google Scholar

Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., et al. (2018). WHO world mental health surveys international college student project: prevalence and distribution of mental disorders. J. Abnorm. Psychol . 127, 623–638. doi: 10.1037/abn0000362

Bakker, D., Kazantzis, N., Rickwood, D., and Rickard, N. (2016). Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Mental Health 3:e7. doi: 10.2196/mental.4984

PubMed Abstract | CrossRef Full Text

Benton, S. A., Heesacker, M., Snowden, S. J., and Lee, G. (2016). Therapist-assisted, online (TAO) intervention for anxiety in college students: TAO outperformed treatment as usual. Prof. Psychol. 47, 363–371. doi: 10.1037/pro0000097

CrossRef Full Text | Google Scholar

Ben-Zeev, D., Schueller, S. M., Begale, M., Duffecy, J., Kane, J. M., and Mohr, D. C. (2014). Strategies for mHealth research: lessons from 3 mobile intervention studies. Administr. Policy Mental Health Mental Health Serv. Res . 42, 157–167. doi: 10.1007/s10488-014-0556-2

Borjalilu, S., Ali Mazaheri, M., and Talebpour, A. (2019). Effectiveness of mindfulness-based stress management in the mental health of Iranian university students: a comparison of blended therapy, face-to-face sessions, and mHealth app (Aramgar). Iran. J. Psychiatry Behav. Sci . 13:84726. doi: 10.5812/ijpbs.84726

Bruehlman-Senecal, E., Hook, C. J., Pfeifer, J. H., FitzGerald, C., Davis, B., Delucchi, K. L., et al. (2020). Smartphone app to address loneliness among college students: pilot randomized controlled trial. JMIR Mental Health 7:21496. doi: 10.2196/21496

Cao, W., Fang, Z., Hou, G., Han, M., Xu, X., Dong, J., et al. (2020). The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res . 287, 1–5. doi: 10.1016/j.psychres.2020.112934

Clough, B. A., and Casey, L. M. (2015a). Smart designs for smart technologies: research challenges and emerging solutions for scientist-practitioners within e-mental health. Prof. Psychol. Res. Pract . 46, 429–436. doi: 10.1037/pro0000053

Clough, B. A., and Casey, L. M. (2015b). The smart therapist: a look to the future of smartphones and mHealth technologies in psychotherapy. Prof. Psychol. Res. Pract . 46, 147–153. doi: 10.1037/pro0000011

Donker, T., Petrie, K., Proudfoot, J., Clarke, J., Birch, M. R., and Christensen, H. (2013). Smartphones for smarter delivery of mental health programs: a systematic review. J. Med. Internet Res . 15:2791. doi: 10.2196/jmir.2791

Field, A. (2009). Discovering Statistics Using SPSS. 3rd Edn . London: SAGE Publications Inc.

Google Scholar

Fish, M. T., and Saul, A. D. (2019). The gamification of meditation: a randomized-controlled study of a prescribed mobile mindfulness meditation application in reducing college students' depression. Simulat. Gaming 50, 419–435. doi: 10.1177/1046878119851821

Fishbein, M., and Ajzen, I. (1975). Belief, Attitude, Intention, and Behavior: An Introduction to Theory and Research . Reading, MA: Addison-Wesley.

Flett, J. A. M., Conner, T. S., Riordan, B. C., Patterson, T., and Hayne, H. (2020). App-based mindfulness meditation for psychological distress and adjustment to college in incoming university students: a pragmatic, randomised, waitlist-controlled trial. Psychology Health 35, 1049–1074. doi: 10.1080/08870446.2019.1711089

Fu, W., Yan, S., Zong, Q., Anderson-Luxford, D., Song, X., Lv, Z., et al. (2021). Mental health of college students during the COVID-19 epidemic in China. J. Affect. Disord . 280, 7–10. doi: 10.1016/j.jad.2020.11.032

Gajecki, M., Andersson, C., Rosendahl, I., Sinadinovic, K., Fredriksson, M., and Berman, A. H. (2017). Skills training via smartphone app for university students with excessive alcohol consumption: a randomized controlled trial. Int. J. Behav. Med . 24, 778–788. doi: 10.1007/s12529-016-9629-9

Haeger, J. A., Davis, C. H., and Levin, M. E. (2020). Utilizing ACT daily as a self-guided app for clients waiting for services at a college counseling center: a pilot study. J. Am. Coll. Health . 12, 1–8. doi: 10.1080/07448481.2020.1763366

Harrer, M., Adam, S. H., Fleischmann, R. J., Baumeister, H., Auerbach, R., Bruffaerts, R., et al. (2018). Effectiveness of an internet- and app-based intervention for college students with elevated stress: randomized controlled trial. J. Med. Internet Res . 20:9293. doi: 10.2196/jmir.9293

Huberty, J., Green, J., Glissmann, C., Larkey, L., Puzia, M., and Lee, C. (2019). Efficacy of the mindfulness meditation mobile app “calm” to reduce stress among college students: randomized controlled trial. JMIR MHealth UHealth 7:14273. doi: 10.2196/14273

Hunt, J., and Eisenberg, D. (2010). Mental health problems and help-seeking behavior among college students. J. Adolesc. Health 46, 3–10. doi: 10.1016/j.jadohealth.2009.08.008

Jackson, D. D., Ingram, L. A., Boyer, C. B., Robillard, A., and Huhns, M. N. (2016). Can technology decrease sexual risk behaviors among young people? Results of a pilot study examining the effectiveness of a mobile application intervention. Am. J. Sexual. Educ . 11, 41–60. doi: 10.1080/15546128.2015.1123129

Johnson, K. F., and Kalkbrenner, M. T. (2017). The utilization of technological innovations to support college student mental health: mobile health communication. J. Technol. Hum. Serv . 35, 314–339. doi: 10.1080/15228835.2017.1368428

Kazemi, D. M., Borsari, B., Levine, M. J., Shehab, M., Nelson, M., Dooley, B., et al. (2018). Real-time demonstration of a mHealth app designed to reduce college students hazardous drinking. Psychol. Serv . 16, 255–259. doi: 10.1037/ser0000310

Kessler, R. C., Amminger, G. P., Aguilar-Gaxiola, S., Alonso, J., Lee, S., and Ustun, T. B. (2007). Age of onset of mental disorders: a review of recent literature. Curr. Opin. Psychiatry 20, 359–364. doi: 10.1097/YCO.0b013e32816ebc8c

Lattie, E., Cohen, K. A., Winquist, N., and Mohr, D. C. (2020). Examining an app-based mental health self-care program, intellicare for college students: single-arm pilot study. JMIR Mental Health 7, 1–15. doi: 10.2196/21075

Lee, R. A., and Jung, M. E. (2018). Evaluation of an mhealth app (destressify) on university students' mental health: pilot trial. J. Med. Internet Res . 5:e2. doi: 10.2196/mental.8324

Leonard, N. R., Silverman, M., Sherpa, D. P., Naegle, M. A., Kim, H., Coffman, D. L., et al. (2017). Mobile health technology using a wearable sensorband for female college students with problem drinking: an acceptability and feasibility study. JMIR MHealth UHealth 5, 1–16. doi: 10.2196/mhealth.7399

Li, S., Wang, Y., Yang, Y., Lei, X., and Yang, Y. (2020). Analysis of influencing factors of anxiety and emotional disorders in children and adolescents during home isolation during the epidemic of novel coronavirus pneumonia. Chin. J. Child Health Care 28, 407–410. doi: 10.11852/zgetbjzz2020-0169

CrossRef Full Text

Linardon, J., Cuijpers, P., Carlbring, P., Messer, M., and Fuller-Tyszkiewicz, M. (2019). The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 18, 325–336. doi: 10.1002/wps.20673

McCloud, T., Jones, R., Lewis, G., Bell, V., and Tsakanikos, E. (2020). Effectiveness of a mobile app intervention for anxiety and depression symptoms in university students: randomized controlled trial. JMIR MHealth UHealth 8, 1–22. doi: 10.2196/15418

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., and The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med . 6:e1000097. doi: 10.1371/journal.pmed.1000097

Newman, M. G., Jacobson, N. C., Rackoff, G. N., Bell, M. J., and Taylor, C. B. (2020). A randomized controlled trial of a smartphone-based application for the treatment of anxiety. Psychotherapy Res . 31, 443–454. doi: 10.1080/10503307.2020.1790688

Ponzo, S., Morelli, D., Kawadler, J. M., Hemmings, N. R., Bird, G., and Plans, D. (2020). Efficacy of the digital therapeutic mobile app biobase to reduce stress and improve mental well-being among university students: randomized controlled trial. JMIR MHealth UHealth 8:17767. doi: 10.2196/17767

Prochaska, J. O., and DiClemente, C. C. (1984). The Transtheoretical Approach: Crossing the Traditional Boundaries of Change . Homewood, IL: J. Irwin.

Rathbone, A. L., Clarry, L., and Prescott, J. (2017). Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: systematic review. J. Med. Internet Res. 19:e399. doi: 10.2196/jmir.8598

Reyes, A. T., Bhatta, T. R., Muthukumar, V., and Gangozo, W. J. (2020). Testing the acceptability and initial efficacy of a smartphone-app mindfulness intervention for college student veterans with PTSD. Arch. Psychiatr. Nurs . 34, 58–66. doi: 10.1016/j.apnu.2020.02.004

Saladino, V., Algeri, D., and Auriemma, V. (2020). The psychological and social impact of covid-19: new perspectives of well-being. Front. Psychol. 11:577684. doi: 10.3389/fpsyg.2020.577684

Shaw, B. M., Lee, G., and Benton, S. (2017). “Work smarter, not harder: expanding the treatment capacity of a university counseling center using therapist-assisted online treatment for anxiety,” in Career Paths in Telemental Health , eds M. Maheu, K. Drude, and S. Wright (Cham: Springer), 197–204. doi: 10.1007/978-3-319-23736-7_19

Spooner, S. E. (2000). “The college counseling environment,” in College Counseling: Issues and Strategies for a New Millennium , eds D. C. Davis and K. M. Humphrey (Alexandria, VA: American Counseling Association), 3–14.

Storrie, K., Ahern, K., and Tuckett, A. (2010). A systematic review: students with mental health problems-A growing problem. Int. J. Nurs. Pract . 16, 1–6. doi: 10.1111/j.1440-172X.2009.01813.x

Taylor, C. B., Fitzsimmons-Craft, E. E., and Graham, A. K. (2020). Digital technology can revolutionize mental health services delivery: the COVID-19 crisis as a catalyst for change. Int. J. Eating Disord . 53, 1155–1157. doi: 10.1002/eat.23300

The Joanna Briggs Institute (JBI) (2017a). Checklist for Randomized Controlled Trials . Adelaide, SA: Joanna Briggs Institute.

The Joanna Briggs Institute (JBI) (2017b). Checklist for Quasi-Experimental Studies . Adelaide, SA: Joanna Briggs Institute.

Torous, J., Nicholas, J., Larsen, M. E., Firth, J., and Christensen, H. (2018). Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid. Based Ment. Health 21, 116–119. doi: 10.1136/eb-2018-102891

Webb, T. L., Joseph, J., Yardley, L., and Michie, S. (2010). Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J. Med. Internet Res . 12, 1–18. doi: 10.2196/jmir.1376

Wilansky, P., Eklund, M., Milner, T., Kreindler, D., Kovacs, T., Shooshtari, S., et al. (2016). Cognitive behavior therapy for anxious and depressed youth: improving homework adherence through mobile technology. JMIR Res. Protoc . 5:e209. doi: 10.2196/resprot.5841

Witkiewitz, K., Desai, S. A., Bowen, S., Leigh, B. C., Kirouac, M., and Larimer, M. E. (2014). Development and evaluation of a mobile intervention for heavy drinking and smoking among college students. Psychol. Addict. Behav . 28, 639–650. doi: 10.1037/a0034747

World Health Organization (2020). Coronavirus Disease (COVID-19) . Available online at: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/question-and-answers-hub/q-a-detail/coronavirus-disease-covid-19 (accessed December, 21, 2020).

Keywords: college students, mental health, mHealth, cognitive-behavioral therapy, counseling services

Citation: Oliveira C, Pereira A, Vagos P, Nóbrega C, Gonçalves J and Afonso B (2021) Effectiveness of Mobile App-Based Psychological Interventions for College Students: A Systematic Review of the Literature. Front. Psychol. 12:647606. doi: 10.3389/fpsyg.2021.647606

Received: 30 December 2020; Accepted: 06 April 2021; Published: 11 May 2021.

Reviewed by:

Copyright © 2021 Oliveira, Pereira, Vagos, Nóbrega, Gonçalves and Afonso. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Carla Oliveira, carlaandreia@ua.pt

This article is part of the Research Topic

Responding to the COVID-19 Pandemic: Health Technology Solutions to Improve Access and Delivery of Cognitive Behavior Therapy

Brief Early Psychological Interventions Following Trauma: A Systematic Review of the Literature

Journal of Traumatic Stress volume  11 ,  pages 697–710 ( 1998 ) Cite this article

163 Accesses

23 Citations

1 Altmetric

Metrics details

A systematic literature search/review was undertaken of brief early psychological interventions following trauma. Only six randomized controlled trials were found, and none of these included group interventions. Of the six trials, two studies associated the intervention with a positive outcome, two demonstrated no difference on outcome between intervention and non-intervention groups, and two showed some negative outcomes in the intervention group. This review suggests that early optimism for brief early psychological interventions including debriefing was misplaced and that there is an urgent need for randomized controlled trials of group debriefing and other early interventions.

This is a preview of subscription content, access via your institution .

Access options

Buy single article.

Instant access to the full article PDF.

Price excludes VAT (USA) Tax calculation will be finalised during checkout.

Rent this article via DeepDyve.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders DSM-IV (4th ed.). Washington D.C.: Author.

Google Scholar  

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders DSM-III-R (3rd ed.). Washington D.C.: Author.

Beck, A., Ward, C., Mendelson, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry , 4 , 561–571.

Birleson, P. (1981). The validity of depressive disorder in childhood and the development of a self-rated scale: A research report. Journal of Child Psychology and Psychiatry , 22 , 73–78.

Bisson, J., & Deahl, M. (1994). Psychological debriefing and prevention of posttraumatic stress: More research is needed. British Journal of Psychiatry , 165 , 717–720.

Bisson, J., Jenkins, P., Alexander, J., & Bannister, C. (1997). A randomised controlled trial of psychological debriefing for victims of acute burn trauma. British Journal of Psychiatry , 171 , 78–81.

Blake, D.D., Weathers, F.W., & Nagy, A.G.Y. (1990). A clinician rating scale for assessing current and lifetime PTSD: The CAPS-1. Behaviour Therapist , 18 , 187–188.

Bordow, S., & Porritt, D. (1979). An experimental evaluation of crisis intervention. Social Science and Medicine , 13a , 251–256.

Brom, D., Kleber, R., & Hofman, M. (1993). Victims of traffic accidents: incidence and prevention of posttraumatic stress disorder. Journal of Clinical Psychology , 49,2 , 131–140.

Bunn, T., & Clarke, A. (1979). Crisis intervention: An experimental study of the effects of a brief period of counselling on the anxiety of relatives of seriously injured or ill hospital patients. British Journal of Medical Psychology , 52 , 191–195.

Caplan, G. (1964). Principles of preventive psychiatry (pp. 26–55). London, UK: Tavistock.

Campbell, D. T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research (p. 2). Chicago: Rand McNally.

Chalmers, T., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., & Ambroz, A. (1981). A method for assessing the quality of a randomized control trial. Controlled Clinical Trials , 2 , 31–49.

Chalmers, I., & Altman, D.G. (Eds.). (1995). Systematic reviews (p. 36). London, UK: British Medical Journal Publishing Group.

Deahl, M., Gillham, A., Thomas, J., Searle, M., & Scrinivasan, M. (1994). Psychological sequelae following the Gulf War: Factors associated with subsequent morbidity and the effectiveness of psychological debriefing. British Journal of Psychiatry , 165 , 60–65.

Derogatis, I., & Melisaratos, N. (1983). The Brief Symptom Inventory: An introductory report. Psychological Medicine , 3 , 595–605.

Dyregrov, A. (1989). Caring for helpers in disaster situations: Psychological debriefing. Disaster Management , 2 , 25–30.

Everly, G.S., & Mitchell, J.T. (1997). Innovations in disaster and trauma psychology. Volume two (pp. 73–87). Ellicott City: Chevron Publishing Corporation.

Foa, E.B., Hearst-Ikeda, D., & Perry, K.J. (1995). Evaluation of a brief cognitive-behavioral program for the prevention of chronic PTSD in recent assault victims. Journal of Consulting and Clinical Psychology , 63 , 948–955.

Goldberg, D., & Hillier, V. (1979). A scaled version of the General Health Questionnaire. Psychological Medicine , 9 , 139–145.

Herman, J.L. (1992). Trauma and recovery (pp. 155–174). New York: Basic Books.

Hobbs, M., & Adshead, G. (1996). Preventive psychological intervention for road crash survivors. In M. Mitchell (ed.), The aftermath of road accidents: Psychological, social and legal perspectives . (pp. 159–171). London, UK: Routledge.

Hobbs, M., Mayou, R., Harrison, B., & Warlock, P. (1996). A randomised trial of psychological debriefing for victims of road traffic accidents. British Medical Journal , 313 , 1438–1439.

Horowitz, M., Wilner, N., & Alvarez, W. (1979). Impact of Events Scale: A measure of subjective stress. Psychosomatic Medicine , 41, 209–218.

Horowitz, M. (1986). Stress response syndromes . Northvale, NJ: Jason Aronson.

Lee, C., Slade, P., & Lygo, V. (1996). The influence of psychological debriefing on emotional adaption in women following early miscarriage: A preliminary study. British Journal of Medical Psychology , 69 , 47–58.

Lundin, T., & Bodegard, M. (1993). The psychological impact of an earthquake on rescue workers: A follow-up study of the Swedish group of rescue workers in Armenia, 1988. Journal of Traumatic Stress , 6 , 129–139.

McFarlane, A.C. (1987). Life events and psychiatric disorder: The role of natural disaster. British Journal of Psychiatry , 151 , 362–367.

McKinley, J. (1981). From ‘Promising Report’ to "standard Procedure’: Seven stages in the career on a medical intervention Milbank Memorial Fund Quarterly/Health and Society , 59 (3), 374–411.

Mitchell, J.T. (1983). When disaster strikes. Journal of Emergency Medical Services , 8 , 36–39.

Morgan, P.P. (1985). The literature jungle. Canadian Medical Association Journal , 134 , 98–99.

Mulrow, C.D. (1995). Rationale for systematic reviews. In I. Chalmers & D.G. Altman (Eds.), Systematic reviews. British Medical Journal , 310 , 1–8.

Myers, D.G. (1989). Mental health and disaster, preventive approaches to intervention. In R. Gist & B. Lubin (Eds.), Psychosocial aspects of disaster (pp. 190–228). New York: John Wiley.

Rachman, S. (1980). Emotional processing. Behaviour Research and Therapy , 18 , 51–60.

Raphael, B., Meldrum, L., & McFarlane, A.C. (1995). Does debriefing after psychological trauma work? British Medical Journal , 310 , 1479–1480.

Reynolds, C., & Richmond, B. (1978). What I think and feel: A revised measure of children's manifest anxiety. Journal of Abnormal Child Psychology, 6, 271–280.

Robinson, R., & Mitchell, J.T. (1993). Evaluation of psychological debriefings. Journal of Traumatic Stress , 6 (3), 367–382.

Rose, S. (1997). Psychological debriefing: history and methods. Counselling—The Journal of the British Association of Counselling , 8(1) , 48–51.

Staab, J.P., Greiger, T.A., Fullerton, C.S., & Ursano, R.J. (1996). Acute stress disorder, subsequent posttraumatic stress disorder and depression after a series of typhoons. Anxiety , 2 , 219–225.

Stallard, P., & Law, F. (1993). Screening and psychological debriefing of adolescent survivors of life-threatening events. British Journal of Psychiatry , 163 , 660–665.

Spielberger, C. (1983). The State Trait Anxiety Questionnaire: A comprehensive bibliography . Palo Alto, CA: California Consultant Psychologist Press.

Taylor, A., & Frazer, A. (1981). Psychological sequelae of operation overdue following the DC-10 air crash in Antarctica. Victoria University of Wellington Publications in Psychology No. 27 . Wellington, New Zealand: Victoria University.

Wessely, S., Rose, S., & Bisson, J. (1997). A systematic review of brief psychological interventions (‘debriefing’) for the treatment of immediate trauma-related symptoms and the prevention of posttraumatic stress disorder (protocol). The Cochrane Library , (CD-ROM) Oxford, UK: Update Software Inc.

Yule, W., & Udwin, O. (1991). Screening child survivors for posttraumatic stress disorder: Experiences from the ‘Jupiter’ sinking. British Journal of Clinical Psychology , 30 , 131–138.

Zigmond, A.S., & Snaith, R.P. (1983). The hospital anxiety and depression score. Acta Psychiatrica Scandinavica , 67 , 361–370.

Download references

Author information

Authors and affiliations.

Cognition, Emotion and Trauma Group, Department of Psychology, University of London, Royal Holloway, Egham, Surrey, TW20 OEX, UK

Suzanna Rose

Gabalfa Clinic, 213 North Road, Cardiff, CF4 3AG, Wales, UK

Jonathan Bisson

You can also search for this author in PubMed   Google Scholar

About this article

Cite this article.

Rose, S., Bisson, J. Brief Early Psychological Interventions Following Trauma: A Systematic Review of the Literature. J Trauma Stress 11 , 697–710 (1998). https://doi.org/10.1023/A:1024441315913

Download citation

Issue Date : October 1998

DOI : https://doi.org/10.1023/A:1024441315913

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Advertisement

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Similar articles being viewed by others

Slider with three articles shown per slide. Use the Previous and Next buttons to navigate the slides or the slide controller buttons at the end to navigate through each slide.

psychological intervention literature

A national experiment reveals where a growth mindset improves achievement

07 August 2019

David S. Yeager, Paul Hanselman, … Carol S. Dweck

psychological intervention literature

Assessment of heterogeneous Head Start treatment effects on cognitive and social-emotional outcomes

19 April 2022

Sun Yeop Lee, Rockli Kim, … S. V. Subramanian

psychological intervention literature

The science of effective learning with spacing and retrieval practice

02 August 2022

Shana K. Carpenter, Steven C. Pan & Andrew C. Butler

psychological intervention literature

Simple questionnaires outperform behavioral tasks to measure socio-emotional skills in students

10 January 2022

Mélusine Boon-Falleur, Adrien Bouguen, … Coralie Chevallier

Schooling and Covid-19: lessons from recent research on EdTech

11 August 2020

Robert Fairlie & Prashant Loyalka

psychological intervention literature

Teaching self-regulation

13 October 2022

Daniel Schunk, Eva M. Berger, … Ernst Fehr

Effectiveness of the flipped classroom model on students’ self-reported motivation and learning during the COVID-19 pandemic

20 July 2021

José María Campillo-Ferrer & Pedro Miralles-Martínez

psychological intervention literature

STEM learning communities promote friendships but risk academic segmentation

20 July 2022

Wesley Jeffrey, David R. Schaefer, … Sabrina Solanki

psychological intervention literature

Remote learning slightly decreased student performance in an introductory undergraduate course on climate change

06 August 2022

Sattik Ghosh, Stephanie Pulford & Arnold J. Bloom

Real-world effectiveness of a social-psychological intervention translated from controlled trials to classrooms

npj Science of Learning volume  7 , Article number:  20 ( 2022 ) Cite this article

1555 Accesses

4 Altmetric

Metrics details

Social-psychological interventions have raised the learning and performance of students in rigorous efficacy trials. Yet, after they are distributed “in the wild” for students to self-administer, there has been little research following up on their translational effectiveness. We used cutting-edge educational technology to tailor, scale up, and track a previously-validated Strategic Resource Use intervention among 12,065 college students in 14 STEM and Economics classes. Students who self-administered this “Exam Playbook” benefitted by an average of 2.17 percentage points (i.e., a standardized effect size of 0.18), compared to non-users. This effect size was 1.65 percentage points when controlling for college entrance exam scores and 1.75 [−1.88] for adding [dropping] the Exam Playbook in stratified matching analyses. Average benefits differed in magnitude by the conduciveness of the class climate (including peer norms and incentives), gender, first-generation status, as well as how often and how early they used the intervention. These findings on how, when, and who naturally adopts these resources address a need to improve prediction, translation, and scalability of social-psychological intervention benefits.

Introduction

“Psychological interventions are done with people, not on people, and these people live in dynamic and diverse social contexts. To predict intervention effects and to advance theory, application, and replicability, we need to understand where and when people will accept the way of thinking put forth by the intervention and be able to use it in their lives to good effect and where and when they will not.” Walton & Yeager (2020, p. 224 1 ).

Policy-makers, educators, psychologists, economists, and school administrators go to painstaking lengths to create interventions to improve student achievement. Some interventions take the form of structural and curricular changes 2 , whereas others are more social-psychological in nature—targeting important mental, emotional, motivational, or social mechanisms of learning, and are often delivered directly to the individual student 3 , 4 . Successful social-psychological interventions have effectively raised the learning, performance, and well-being of tens of thousands of students across the achievement spectrum in rigorous double-blind, randomized controlled trials (RCTs) 5 , 6 , 7 , 8 , 9 , 10 .

The end goal of developing and rigorously testing these psychological interventions in RCTs is to provide them as effective psychological resources to students, parents, and educators. While there are multiple ways to scale social-psychological interventions (such as offering them to teachers to use as pedagogical tools, conducting school-wide workshops, or incorporating intervention content into the curriculum), social-psychological interventions that students can self-administer to help themselves are especially important and relevant in secondary, post-secondary, and online education, where learning is largely self-regulated.

Once tried-and-tested social-psychological interventions are widely distributed “into the wild” for students to self-administer, does their use still predict academic achievement? What kinds of students take up these interventions on their own, and how effectively do they use the resources? Under what conditions are they more or less effective? These are important scientific questions about the translational effectiveness of self-administered interventions (including translational effect sizes, user uptake, and heterogeneity).

To our knowledge, unlike school-wide program evaluations, there are no rigorous, large-scale naturalistic examinations of such effectiveness of student-level social, affective, or motivational interventions, after they are distributed for students to adopt on their own. There already exists many examples of efficacious social-psychological interventions 3 , established through gold-standard laboratory and classroom experiments—such as the Strategic Resource Use intervention, which was tested in two RCTs with relatively large effect sizes on course grades 6 ; the values affirmation intervention, which was replicated in many RCTs across different age groups and school sites 11 , 12 ; the social-belonging intervention, which produced performance and health benefits among minorities across multiple RCTs, including a large field experiment across 21 colleges and universities 1 , 9 ; and the growth mindset intervention, which robustly replicated over many experiments and was recently tested in a randomized, controlled trial using a nationally representative sample of 65 U.S. high schools 10 . Some of these interventions have even been tested cross-culturally via Massive Open Online Courses (MOOCs) 13 , 14 .

These rigorous experiments robustly demonstrated efficacy —but speak less to the interventions’ effectiveness after being released “into the wild” for learners to adopt and self-administer as a self-regulatory choice. We borrow these terms from the medical and public health literature, where efficacy studies establish scientific validity in controlled experiments, whereas effectiveness studies test external validity when the intervention is delivered in the real world 15 , 16 . Compared to efficacy trials, the success of effectiveness studies like ours depend on multiple factors, including but not limited to the efficacy and accessibility of the intervention, along with participants’ receptivity to, willingness to engage in, and effective use of the intervention 15 .

Despite their importance, efficacy trials alone are considered “necessary… but not sufficient for, effectiveness” in the real world ( 15 , p. 455), where people have the choice of using or not using any resource amongst many others available. To understand what keeps psychologically wise interventions effective for student learning when they progress from an experimental trial to a freely available resource, scientific inquiry must go beyond controlled experiments 16 , 17 . Some social-psychological interventions may have been beneficial under relatively controlled conditions when the intervention was imposed upon students, often in a highly structured manner (i.e., students have no choice about which treatment to engage in or when they do so, but are assigned to either a treatment or control, and directed when and how to engage with the materials). But in real-world learning, especially starting from mid- to late-adolescence where learning becomes increasingly self-managed, there is no guarantee that students would make use of a particular intervention among many alternative resources, that students would even use the intervention effectively on their own, or that intervention use would relate to academic performance.

Hence, especially for interventions that will eventually be given to students to use as tools for learning, it is important to conduct naturalistic studies of translational effectiveness that follow-up and complement efficacy-focused experiments. Such studies provide much-needed information about (a) whether self-administration of the intervention relates to academic achievement for most students, (b) under what conditions using the intervention confers more versus less benefits, and (c) how intervention use and benefits may differ across different kinds of learners.

To study these questions, we adapted a previously-validated Strategic Resource Use intervention, which was designed to increase students’ self-reflection about their resource use 6 , into an online app called the “Exam Playbook.” We chose the Strategic Resource Use intervention because it had previously been experimentally tested and found efficacious at raising students’ course grades by an average of one-third of a letter grade in two double-blind RCTs (total N  = 361; effect size Cohen’s d  = 0.33 and 0.37) at the same university 6 , 18 ; it was online and self-administered, and therefore could be conveniently modified for testing across a variety of classes; and it benefitted diverse demographic groups of students in prior experiments 6 . Moreover, the university administration was supportive of widely distributing and testing the use of this intervention as a free resource for students.

The Exam Playbook was made user-friendly and engaging for the average student. Similar to its predecessor 6 , the Exam Playbook guided students’ self-reflective resource use when preparing for an upcoming exam. It prompted students to anticipate the format and demands of their upcoming exam, provided a comprehensive checklist of learning resources available in the class, and asked students to select which would be useful in helping them study for the exam. It asked students to explain why each resource they had chosen would be useful to their learning (to make clear the purpose for which they would use the resource), and then to plan out when, where, and how they were going to use the resources chosen (which increases the likelihood of follow-through on their plans 19 ).

We used cutting-edge “ECoach” technology, developed within the past 6 years, to innovate on class-level customizations to the Exam Playbook (e.g., resource checklist, distribution timing)—making it highly scalable across diverse class contexts (see Methods for details; 20 , 21 ). Through ECoach, we made the intervention freely available to college students enrolled in 14 large introductory STEM classes across 2 semesters at a large public U.S. university. Students could freely choose and access our intervention, as one of many possible learning resources (behaviors which ECoach tracked). This allowed us to collect behavioral data about who accessed the intervention and when they did so.

Combining the Exam Playbook with ECoach technology, our translational study focused on estimating intervention effectiveness and understanding the factors that can influence intervention effects, when a social-psychological intervention for learning (in this case, the Strategic Resource Use intervention) is released “into the wild,” where students actively manage their own learning 17 . We tested our hypothesis that, on average, we would observe a statistically-significant relation between using the intervention and students’ exam performance across classes, but that it would be smaller in magnitude than the effect sizes observed in the RCTs ( d s = 0.33, 0.37; 22 ). For reference, a difference of 0.2 is considered a large difference in field research on factors that predict educational outcomes, especially when an intervention is low-cost and scalable 10 , 23 , 24 . Beyond testing the main effect of the intervention, we planned additional exploratory analyses to investigate heterogeneity by classes and student demographics.

There are multiple benefits to having this translational effectiveness study complement prior efficacy RCTs: One, we can estimate the effectiveness of the intervention when students autonomously self-administer it 16 . Two, we can capture what kinds of students are utilizing the intervention in the real world under realistic learning conditions, and how they are making use of it (e.g., timing, dosage) 25 , 26 . Three, we supplement the previous RCTs 6 by showing that an intervention tested on a small scale can be effective when distributed on a large scale. Four, we are able to test possible heterogeneity in intervention-use effectiveness across diverse classes. From a policy standpoint, this study of effectiveness is a crucial step when going from bench to bedside—or from “ controlled trial to classroom ” in the education context—especially for interventions that could be widely distributed to hundreds or thousands of students.

We examined 12,065 students’ use (versus non-use) of the Exam Playbook across 14 introductory STEM and Economics classes over 2 consecutive (Fall and Winter) semesters. The 7 courses included in each semester were: Introductory Statistics, Introductory Biology, General Chemistry, General Physics, Introductory Programming (for Engineers), Introductory Programming (for Programmers), and Introductory Economics. A breakdown of sample demographics is presented in Supplementary Table 1 .

Across both semesters, on average, 43.6% ( SD  = 29.3%; range: 5.6–91.4%) of students in each class engaged with the Exam Playbook at least once. We operationalized a “use” of the Exam Playbook to mean accessing and completing the intervention, which includes: completing the resource checklist, explaining why each resource would be useful, and planning resource use. That is, students had to click through to the end of the intervention to be counted as having used it (Supplementary Note 1 contains further details about how we defined and operationalized “use”). Apart from varying across classes, Exam Playbook use also varied between exams, as a student might choose to use it on one exam but not another. Note that the original intervention was only offered before 2 exams (i.e., 2 doses maximum), but in this translational study, it was offered before all available exams in each class, which could differ by class (with the exception of Physics Exam 4 when it was not offered). Table 1 gives a detailed breakdown of the number of times the Exam Playbook was offered and used on each exam across the different classes.

Does self-administration of the exam playbook predict exam performance?

We tested the hypothesis that using the Exam Playbook benefits students' exam performance, by comparing the average exam scores of students who used the Exam Playbook at least once in the class with students who did not use the Exam Playbook at all. Following recent recommendations in statistics and psychological science to move toward a focus on effect-size estimation 27 , 28 , we ran a "mini meta-analysis" 29 across the 14 classes using a random-effects meta-analysis model 30 , treating each class as a separate "experiment" and with a mind towards analyzing heterogeneity across classes. This allowed us to estimate the generalizability of the effect across classes, as well as the variation due to inter-class differences—both of which are important for understanding how the Exam Playbook can benefit future students in various subjects.

Our meta-analysis, summarized in Fig. 1 , revealed that students who used the Exam Playbook in their class scored 2.17 ([95% CI: 1.13, 3.21], p  < 0.001) percentage points higher than non-users, for their average exam score (normalized and upon 100 percentage points). To put this effect size into context, a 2.17 percentage point difference translates to a standardized difference (Cohen’s d ) of 0.18—a substantial effect for a free, highly scalable, and self-administered intervention. As mentioned earlier, a difference of 0.2 is considered a large difference in field research on factors that predict educational outcomes, especially for low-cost and scalable interventions 10 , 23 , 24 . As Fig. 1 shows, the effect was positive in 13 out of 14 classes, and there was a high correlation of r  = .87 ( p  = 0.010) between the effect sizes for each class across both semesters.

figure 1

Note. Forest plot summarizing a meta-analysis of the effect of using the Exam Playbook on students’ averaged exam score. Data points represent the effect size for each class in each semester, with error bars representing 95% confidence intervals. The diamond in the last row represents the weighted meta-analytic effect size 30 , and corresponds to a standardized effect size (Cohen’s d ) of 0.18.

Two robustness checks further validated these results: One, controlling for students’ college entrance exam scores as a covariate (students in our sample were mostly freshmen who did not yet have college GPA), the overall meta-analytic trend remained consistent: Exam Playbook users scored an average of 1.65 ([0.55, 2.75], Cohen’s d  = 0.14, p  = .003) percentage points higher than non-users on their average exam score. We tested demographic factors (gender, race and first-generation status) as potential moderators later in the Results. Two, to supplement our class-level analyses, our results held when we examined Exam Playbook use on performance at the exam-level within class. A mixed-effects meta-analysis (with exam as a fixed effect within each class, and class as a random effect) across all 40 exams observed showed that students who used the Exam Playbook on a given exam scored an average of 2.91 ([1.81, 4.01], Cohen’s d  = 0.22, p  < 0.001) percentage points higher than students who did not use the Exam Playbook on a given exam.

Under what class conditions might the exam playbook be more or less effective?

As shown in Fig. 1 , there was substantial heterogeneity in the estimated effect size of using the Exam Playbook across different classes. The average effect size was largest in the Introductory Statistics course (5.18 percentage points in Fall and 6.74 in Winter), which was the exact course for which the original intervention was designed and experimentally tested 6 . Thus, this serves as an assessment of the effectiveness of the intervention when made freely available within the same class context (c.f. an RCT-based efficacy effect size of 3.64 and 4.21 percentage points in two studies in 6 ).

The other courses allow us to examine the generalization of the Exam Playbook to different class contexts. As a conservative test of the generalizability of Exam Playbook use on exam performance beyond the Introductory Statistics course, we repeated our analyses using only the 6 other courses (12 classes total) excluding Introductory Statistics. On average, using the Exam Playbook still conferred benefits to students in these courses. The meta-analytic effect size was smaller and still significant: students who used the Exam Playbook scored an average of 1.60 ([1.00, 2.19], d  = 0.13, p  < 0.001) percentage points higher than non-users. When controlling for college entrance exam scores, we observed a 1.07 percentage points difference ([0.29, 1.85], d  = 0.09, p  = 0.007).

After Introductory Statistics, which had the highest use rates and effect sizes, students in the two Introductory Programming courses enjoyed the next-largest average benefits—2.24 percentage points averaged across both semesters and both programming courses (we note that the Introductory Economics course had substantial differences in effect sizes and uptake across Fall and Winter semesters). On the other end of the spectrum, the smallest average effect sizes from using the Exam Playbook were observed in the General Physics and General Chemistry courses (0.12 percentage points averaged across both semesters for General Physics; 0.74 percentage points for General Chemistry).

One plausible reason for such heterogeneity at the class level could be how much the climate of the course supported such strategic resource use, including Exam Playbook use. According to contemporary theorizing about psychological intervention effect heterogeneity, “change requires planting good seeds (more adaptive perspectives)… in fertile soil (a context with appropriate affordances)” ( 1 , emphasis ours). That is, perhaps the Exam Playbook was more useful to students who were in course climates more conducive to the psychology of the Exam Playbook.

Two possible operationalizations of this course climate (at the class-level) are peers’ uptake of the Exam Playbook 10 , 31 and teachers’ degree of support toward engaging in the Exam Playbook as a useful learning resource 21 —both of which reflect powerful social norms that could influence students’ engagement with and degree of benefit from the Exam Playbook 1 , 10 , 32 .

We fit two separate linear models using (a) the average Exam Playbook usage (by course) and (b) the quantifiable presence/absence of extra course credit offered for engaging in the Exam Playbook, to predict the effect size for each class. Instructors in 4 of the 7 courses (specifically Introductory Statistics, Introductory Biology, Introductory Programming (Programmers), and Introductory Programming (Engineers)) incentivized the use of the Exam Playbook by offering bonus credit to students' final course grade for using it. Importantly, however, these bonuses did not influence our main outcome measure: exam performance.

Indeed, the average Exam Playbook usage in a class (the peer norm) was positively associated with the effect size of using the Exam Playbook ( b  = 2.49 [1.82, 3.16], d  = 0.20, p  < 0.001). Similarly, teacher support in the form of course credit incentives offered related to a larger effect size than when it was not offered ( b  = 2.04 [0.25, 3.84], d  = 0.17, p  = 0.046).

Could differences in the extensiveness of resources provided or the kinds of resources most students selected to use (such as practice-based versus simple reading and memorization) have explained the variation in effect sizes across classes? Our data did not support either of these possibilities: the number of resources offered varied only slightly among classes (range: 11–15), and the types of resources that students selected the most for use were generally similar across classes (see Supplementary Note 2 ). Hence, we ruled out that that either of these factors strongly explained class-level heterogeneity.

Intra-individual changes in exam performance when dropping vs. adopting the exam playbook

One difficulty of observational (effectiveness) studies, compared to experimental (efficacy) studies, is teasing apart the effects of confounding variables. Methods such as matching and difference-in-difference modeling try to control for these effects. We conducted two analyses based on matching, to examine how intra-individual variation in Exam Playbook usage tracked changes in academic performance. We matched students using their background and behavior in the initial portion of the class, and then examined how subsequent behavior tracked exam performance.

In these classes, there were natural variations in Exam Playbook usage. Some students started off not using the Exam Playbook, and picked up (or “adopted”) the Exam Playbook on later exams, while others used the Exam Playbook early on but dropped it later in the class (see Supplementary Table 2 for descriptives). These natural covariations allowed us to assess the average effect of “adopting” and “dropping” the Exam Playbook within individuals. If Exam Playbook usage benefits students’ performance, we should expect their exam performance to covary with students’ Exam Playbook usage patterns—with “adopting” and “dropping” associated with increased and decreased exam performance, respectively.

Using stratified matching 33 , we matched these students on their initial exam performance (the first exam in the class), college entrance scores, gender, race, and first-generation status, and estimated the average effect of adopting and dropping the Exam Playbook on their subsequent exams. Because most of the activity of Exam Playbook usage within a class occurred within the first two exams of the class (94%), we restricted this analysis to only the first two exams of each class. Stratified matching analysis was performed for each class separately (13 classes; the Introductory Economics Winter class did not have sufficient sample size for stratified matching) and we computed a meta-analytic estimate using a mixed-effects meta-analysis.

To estimate the average effect of adopting the Exam Playbook, we took the subset of students who did not use the Exam Playbook on their first exam. Of these, some students adopted the Exam Playbook on their second exam, while others did not. When matched on their first exam performance, college entrance scores, and demographics, students who adopted the Exam Playbook performed an average of 1.75 percentage points ([0.69, 2.81], d  = 0.12, p  = .001) better on the second exam, compared to those who never used it (Fig. 2 left panel).

figure 2

Note. Forest plot showing effect sizes from stratified matching analyses. Numbers below each course name indicate the number of students in that analysis (and as a percentage of the total class). Left: Effect of “adopting” the Exam Playbook. Both groups did not use the Exam Playbook at Exam 1; students who used it on Exam 2 outperformed students who did not. Right: Effect of “dropping” the Exam Playbook. Both groups used the Exam Playbook for Exam 1; students who dropped the Exam Playbook at Exam 2 did worse than students who consistently used it. Error bars reflect 95% confidence intervals.

To estimate the effect of dropping the Exam Playbook, we repeated this analysis on the subset of students who had used the Exam Playbook for their first exam. Of these students, some dropped the Exam Playbook on their second exam, while others continued using it. When matched on their first exam performance, college entrance scores, and demographics, students who dropped the Exam Playbook performed an average of 1.88 percentage points ([0.64, 3.11], d  = 0.14, p  = .003) worse , compared to those who kept using it (Fig. 2 right panel).

Following our earlier conservative test of generalizability beyond Introductory Statistics, repeating this stratified matching analyses with the 6 other courses excluding Introductory Statistics, we still observed these effects of adopting and dropping the Exam Playbook—albeit with smaller effect sizes. When matched on their first exam performance, college entrance scores, and demographics, students who adopted the Exam Playbook performed an average of 1.56 percentage points ([0.47 2.65], d  = 0.10, p  = .005 better on the second exam, compared to those who never used it. When matched on their first exam performance, college entrance scores, and demographics, students who dropped the Exam Playbook performed an average of −1.53 percentage points ([−3.29, 0.22], d  = −0.12, p  = .087) worse, compared to those who kept using it (although this smaller effect of dropping was not significant at the 0.05 level).

Overall, these intra-individual data add further evidence to our meta-analyses suggesting that, on average, using the Exam Playbook predicts exam performance. We describe in Supplementary Note 3 that these results also replicate using a difference-in-difference analytical method.

Dosage and timing

Next, we examined whether there were dosage and timing effects of using the Exam Playbook. Uptake of the Exam Playbook peaked between the first two exams, and then dropped thereafter if there were more than 2 exams in the course (see Table 1 ). Mixed-effects meta-analyses indicated that using the Exam Playbook on more occasions (i.e., higher dosages) related to better average exam performance ( b  = 2.18 percentage points [1.18, 3.19], d  = 0.18, p  < 0.001) among students who used the Exam Playbook—consistent with findings from the original efficacy experiments 6 .

The Exam Playbook was made available to students up to 10 days prior to their exams. The average student who used the Exam Playbook engaged with it a week ( M  = 7.0 days, sd  = 3.0 days) before their exams. We used time of usage (number of days before the exam) to predict exam performance at the exam-level. Students who used the Exam Playbook benefited more from using it earlier ( b  = 0.42 percentage points per day [0.29, 0.54], d  = 0.03 per day, p  < 0.001). This suggests that early preparation is associated with better Exam Playbook effectiveness, although it could also reflect other motivation-relevant traits like better time-management and general self-regulatory ability 34 . For example, students who used the Exam Playbook very close to the exam date might have procrastinated or crammed their exam preparation—reflecting lower self-regulation 35 .

What kinds of students naturally used the exam playbook? Were there differential benefits to different groups of students?

To better understand which students naturally used the Exam Playbook as a learning resource, we ran a mixed-effects logistic regression using academic ability (college entrance exam score) and demographic variables (gender, race, first-generation status) as predictors of whether students used the Exam Playbook at least once in their classes. Academic ability did not significantly predict Exam Playbook usage ( χ 2 (1) = 0.24, p  = .621), which suggests that natural adoption of this Exam Playbook resource may not have been restricted to higher performers or simply more motivated students. However, there were demographic differences in natural uptake of the Exam Playbook. Gender significantly predicted Exam Playbook adoption ( χ 2 (1) = 196.18, p  < .001): the odds of females using the Exam Playbook were 2.22 times higher than males. Race also predicted Exam Playbook adoption ( χ 2 (7) = 21.78, p  = .003): in particular, Black and Hispanic students were less likely to use the Exam Playbook on their exams (Black students had 0.65 times the odds of using it compared to White students, p  = .003, and 0.56 times the odds compared to Asian students, p  < .001; Hispanic students had 0.79 times the odds of using it compared to White students, p  = .026, and 0.68 times the odds of using it compared to Asian students, p  < .001). First-generation status did not predict Exam Playbook adoption ( χ 2 (1) = 0.79, p  = .373).

Could certain groups of students have benefitted more (or less) from using the Exam Playbook? We fitted separate mixed-effects linear models to test the moderation effect of gender, race, and first-generation status on the effectiveness of using the Exam Playbook. Gender significantly moderated Exam Playbook effects: while females generally performed worse than males ( b  = −3.83 [−4.50, −3.17], d  = 0.30, p  < .001), as is commonly observed in STEM classes, female users benefitted 2.35 percentage points ( b  = 2.35 [1.45, 3.26], d  = 0.19, p  < .001) more from using the Exam Playbook than male users—a substantial 61.4% reduction in the gender gap. Race did not moderate Exam Playbook effects ( χ 2 (7) = 6.11, p  = .527). First-generation status significantly moderated Exam Playbook effects: while first-generation students generally performed worse than non-first-generation students ( b  = −7.04 [−7.95, −6.12], d  = 0.57, p  < .001), using the Exam Playbook reduced this gap by an average of 2.25 [0.96, 3.54], d  = 0.18, p  < .001, percentage points—a 32.0% reduction in the first-generation achievement gap.

Recent discussions on non-replications and lack of implementation fidelity when practitioners try to execute social-psychological interventions themselves 36 , 37 , 38 , 39 suggest that more rigorous effectiveness tests are needed. Social-psychological interventions that target the social, affective, or motivational mechanisms of learning can be efficacious in rigorous laboratory or field trials, but still need to be further tested for their effectiveness when released for self- or facilitated-administration. Granted, not all social-psychological interventions are meant to be self-administered by students—but where they are and can be after distribution, it is worthwhile to systematically track and understand their use and benefits “in the wild.”

Our research provides an example of a large-scale, systematic effectiveness test of an efficacious intervention, addressing crucial empirical questions about its benefits, boundary conditions, users, and self-administered timing and dosage. We emphasize that such effectiveness research does not merely apply intervention design to practice in an atheoretical manner—instead, it importantly informs how scientists should think about intervention design and testing, along with the myriad factors that affect its translational effectiveness in actual classrooms (e.g., classroom climate, student demographics, timing, dosage). By identifying possible boundary conditions and other sources of intervention heterogeneity, this work is a step toward building better theories of the contextual factors and psychological mechanisms that matter for self-administered effectiveness—theories that future research could systematically test with additional measures of such contextual differences and psychological states 22 , 40 .

Building on earlier RCT causal evidence 6 , the purpose of this research was to scale, examine who takes up a freely available intervention resource, and to investigate its heterogeneity and generalizability. To minimize the limitations of drawing inferences from correlational data, we presented converging evidence on the potential benefits of Exam Playbook use from multiple analytical approaches—including estimating the meta-analytic effect size at both the class ( d  = 0.18) and exam levels ( d  = 0.22), a robustness test that controlled for prior academic performance ( d  = 0.14), stratified matching analysis ( d  = 0.12/0.13) for adding/dropping the Exam Playbook between exams, and difference-in-difference modeling ( d  = 0.16/0.12 for adding/dropping; Supplementary Note 3 ). The effects observed here (using both inter- and intra-individual modeling) are consistent with previous RCT evidence, showing that greater Exam Playbook usage relates to higher academic performance, even when controlling for obvious third variables.

Moreover, these observed benefits associated with Exam Playbook use were not simply due to students’ concurrent use of other learning resources available on ECoach (e.g., grade calculator, “to do list” 21 ;). Other research on general ECoach use and engagement, conducted with a separate sample of students across 5 courses, found that using the Exam Playbook significantly and uniquely predicted course performance, even when controlling for the use of other ECoach resources 21 .

We tested for possible heterogeneity in the self-administered intervention effects, with a primary interest in understanding how the class climate might relate to students’ accrued benefits. Using the Exam Playbook seems to be more useful when more classmates use it and when teachers encourage its use—in other words, class norms supporting strategic resource use matter. Teachers could proactively encourage and nurture the psychology of strategic resource use in their courses—such as by incentivizing the use of the Exam Playbook, by encouraging groups of students to work together on the Exam Playbook, or by incorporating self-regulated resource use into their teaching.

There is also the possibility that the when individual students use the intervention, they learn to value and engage in self-regulated resource use to a greater degree. This intra-individual change can contribute to ecological change at the classroom level 31 , 40 , creating a learning environment with norms that value and support engagement in self-regulated resource use. Such bidirectional effects of intra-individual effects on classroom climate and classroom climate on students’ benefits are worth future investigation, because they shape how we understand where an intervention will effectively take root and how its effects might perpetuate 3 . Future research could also examine whether individual differences 41 , course structures, curricula, or demographic make-ups may be associated with greater (or less) intervention benefits.

Of secondary interest, we also tested for and discovered differences in Exam Playbook uptake by gender, race, and first-generation status. Compared to males, female students tend to be more conscientious, and may naturally be drawn toward organizing and planning their learning 42 , 43 , 44 , 45 , which the Exam Playbook facilitates. Hence, they also tend to benefit more from its use. These results suggest that Exam Playbook adoption could potentially help reduce the gender gap in STEM classes—an idea that intervention research should systematically investigate in an RCT. Although first-generation status did not predict Exam Playbook usage, first-generation student users did benefit more than non-first-generation students from using the resource—suggesting that we could encourage greater Exam Playbook adoption among first-generation students to promote their self-regulation and learning.

Black and Hispanic students were less likely than White and Asian students to use the Exam Playbook, even when it was freely available. It could be that these students experience greater identity threat in some of their STEM classes, which may undermine their motivation to engage in their classes and with resources provided for their learning 46 . Future research could pair threat-reducing interventions (such as values affirmation and belonging interventions) with the Exam Playbook to test if this might pave the way for greater use and benefits among these minority groups.

This research is among the first to follow an RCT-validated social-psychological intervention through effectiveness testing, after it is released for students’ self-administration. It demonstrates an example of successful scaling and generalizability of a class-tailored intervention; and highlights the importance of class climate, self-administered timing and dosage, and student background in explaining heterogeneity in uptake and benefits. We hope this will encourage even more effectiveness research at scale on how people adopt and benefit from social-psychological interventions, when given the free choice to use it or not.

We adapted the Exam Playbook from the original Strategic Resource Use intervention, and delivered it using ECoach technology to multiple classes. ECoach enabled us to tailor its content (e.g., set of resources described, exam reminder) and delivery (timing of the intervention delivery before exams, total possible dosages offered) to each class. This study was approved as exempt from further oversight by the University of Michigan Institutional Review Board (IRB #HUM00119869). The research reported here was conducted as secondary data analysis, and under FERPA exception for educational research, given that the use of the ECoach platform (and the Exam Playbook feature) is now a standard part of the institution’s educational practices.

ECoach technology for class-specific tailoring and delivering the intervention at scale

At our test university, ECoach technology is widely used (it currently has 24,165 users in 2021), and complements the university’s Learning Management System as a source of academic advising and various learning resources 20 , 21 , 47 . To deliver the Exam Playbook to the 14 STEM classes in our study, we leveraged this cutting-edge technology that was familiar and easily accessible to students.

The Exam Playbook was housed within ECoach as one of many available learning resources that students could choose to use (or not) autonomously. This approximates actual college learning, where students often have many resources (e.g., course packs, textbooks, peer study groups, library books, teacher office hours, online discussion forums) that they can choose to use or not for their learning 48 . This enabled us to test whether and how students naturally use the Exam Playbook when it is freely available as a learning resource, among many others, rather than when it is one specifically isolated resource imposed upon them in an RCT.

For each of the 14 STEM classes, psychologists, designers, and instructors collaborated to customize class-specific parts of the Exam Playbook, such as the checklist of available resources, and tailored exam reminders. As mentioned earlier, students received a personalized reminder via ECoach that the Exam Playbook was available before each of their course exams. This reminder was delivered on the online ECoach website, through email, or through text message, depending on the student’s notification preferences in ECoach.

ECoach automatically tracked and organized students’ (a) use of and responses to the Exam Playbook, (b) course exam performance data from the University’s Learning Management System, and (c) registrar data (e.g., prior performance, demographics). These data allowed us to test our research questions, described above.

Exam playbook

Students were informed via ECoach personalized messaging that the Exam Playbook was available to them as an exam preparatory resource to use if they wished. As in previous RCTs 6 , access to the pre-exam exercise was officially made available 10 days prior to an exam. This was customized according to the timing and number of the exams in each course (see Supplementary Note 4 for more details about timing). For example, students in the Introductory Statistics course had the Exam Playbook available for use before each of their 3 exams, and they were sent a message via ECoach about this available resource 10 days before each exam. Importantly, students could decide for themselves whether or not to use the Exam Playbook, and this resource was provided alongside a list of other online learning resources on ECoach that were also freely available to students. To complement the earlier description of the Exam Playbook, we provide example screenshots of key components of the Exam Playbook in Supplementary Note 5 . At the end of the Exam Playbook, students were offered a summary of their responses (including the resources they selected, their reasons why each resource would be useful, and their plans) to print out and keep if they chose.

The 7 courses that were involved in our study across 2 consecutive semesters included: Introductory Statistics, Introductory Biology, General Chemistry, Introductory Economics, Introductory Programming (for Programmers), Introductory Programming (for Engineers), and General Physics. All except Introductory Economics are officially considered large introductory STEM courses.

Statistical approach

Our analysis strategy involved computing effects within each of the 14 classes we observed (7 courses x 2 semesters), which themselves have between 1 to 4 exams. Then, treating each class as a separate “experiment”, we would compute a meta-analytic effect size using a random-effects meta-analysis model 30 . We took this general approach to all our analyses. Meta-analysis estimates were computed using the meta package (v4.18-1 49 ) in R (see Supplementary Note 6 for replication using hierarchical linear modeling, and Supplementary Note 7 for R code for all of our models).

Treatment effect of exam playbook

For each class, we predicted students’ average exam performance using a binary predictor that indicated whether the student had used the Exam Playbook at least once in the class, operationalized as logging into the Exam Playbook and fully clicking through the complete intervention. We then aggregated the estimates from the 14 individual models, weighting them using their standard errors.

For the first robustness check, we added college entrance exam scores as a covariate. For the second robustness check, we repeated this analysis at the exam level. That is, we predicted exam score using a binary predictor whether the student used the Exam Playbook on that particular exam. We then aggregated the exam effects into a class effect, and then aggregated the effects across classes.

Class heterogeneity analysis

We predicted the Exam Playbook effect size of each class using the proportion of Exam Playbook usage in the class (i.e., proportion of students that used the Exam Playbook at least once, from 0 to 1) and a binary predictor indicating whether extra course credit was offered for using the Exam Playbook.

Stratified matching analysis

We performed stratified matching using the MatchIt package (v4.2.0 50 ). Because of the steep drop-off in Exam Playbook usage after the first two exams, we focused on our analyses on Exam Playbook usage and exam performance on the first two exams (see Supplementary Note 8 for a discussion of this cut-off, including background, plausible explanations, and future directions). This analysis first computes a propensity score by using the covariates (previous exam score, college entrance score, gender, race, and first-generation status) to predict the treatment group (e.g., adopted the Exam Playbook versus not) via logistic regression. It then stratifies the propensity scores based on five quantiles. Based on these strata, the final regression model is weighted to give an estimate of the Average Treatment Effect (ATE) on the performance on the second exam. This analysis was run on each class separately. The aggregated estimated was computed via random-effects meta-analysis (using the meta package like above).

We fit linear models for each class before estimating an aggregate effect using random-effects meta-analysis. To estimate the dosage effect, we considered the subset of Exam Playbook users, and used the number of times they used the Exam Playbook to predict their average exam score in the class.

To estimate how timing of usage affects exam performance, we again considered the subset of Exam Playbook users, but now examined performance on each individual exam. We defined a variable, “time_left,” which counts the number of days between the Exam Playbook usage and the exam itself.

Moderation of Exam Playbook usage and effects

To test for self-selection, we predicted whether a student engaged with the Exam Playbook at least once in the class, using as predictors their college entrance scores, gender, race, and first-generation status. Similar to previous analyses, this analysis was performed separately for each class and aggregated using random-effects meta-analysis.

To estimate if the Exam Playbook effect size is moderated by gender, race, and first-generation status, we tested (separately in three models) the interaction of Exam Playbook usage with gender, race, and first-generation status. To compute first-generation status from the available registrar data, we classified students as “first-generation” if their parents had not received a college degree or above.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

The data is protected under the Family Educational Rights and Privacy Act (FERPA) and any access to the underlying data is contingent on approval from the University of Michigan, per FERPA guidelines and regulations. Requests for student data should be sent to the Office of Enrollment Management at [email protected]

Code availability

Analysis code is available on Open Science Framework at https://osf.io/6qej7/ 51 .

Walton, G. M. & Yeager, D. S. Seed and soil: psychological affordances in contexts help to explain where wise interventions succeed or fail. Curr. Directions Psychol. Sci. 29 , 219–226 (2020).

Article   Google Scholar  

Hattie, J. Visible Learning: A synthesis of over 800 meta-analyses relating to achievement . (Routledge, London, England, 2009).

Google Scholar  

Walton, G. M. & Wilson, T. D. Wise interventions: psychological remedies for social and personal problems. Psychol. Rev. 125 , 617–655 (2018).

Article   PubMed   Google Scholar  

Yeager, D. S. & Walton, G. M. Social-psychological interventions in education: they’re not magic. Rev. Educ. Res. 81 , 267–301 (2011).

Brady, S. T., Cohen, G. L., Jarvis, S. N. & Walton, G. M. A brief social-belonging intervention in college improves adult outcomes for Black Americans. Sci. Adv. 6 , eaay3689 (2020).

Article   PubMed   PubMed Central   Google Scholar  

Chen, P., Chavez, O., Ong, D. C. & Gunderson, B. Strategic resource use for learning: a self-administered intervention that guides self-reflection on effective resource use enhances academic performance. Psychol. Sci. 28 , 774–785 (2017).

Cohen, G. L., Garcia, J., Apfel, N. & Master, A. Reducing the racial achievement gap: a social-psychological intervention. Science 313 , 1307–1310 (2006).

Article   CAS   PubMed   Google Scholar  

Paunesku, D. et al. Mind-set interventions are a scalable treatment for academic underachievement. Psychol. Sci. 26 , 784–793 (2015).

Walton, G. M. & Cohen, G. L. A brief social-belonging intervention improves academic and health outcomes of minority students. Science 331 , 1447–1451 (2011).

Yeager, D. S. et al. A national experiment reveals where a growth mindset improves achievement. Nature 573 , 364–369 (2019).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cohen, J., McCabe, L., Michelli, N. M. & Pickeral, T. School climate: research, policy, practice, and teacher education. Teach. Coll. Rec. 111 , 180–213 (2009).

Miyake, A. et al. Reducing the gender achievement gap in college science: a classroom study of values affirmation. Science 330 , 1234–1237 (2010).

Kizilcec, R. F., Pérez-Sanagustín, M. & Maldonado, J. J. Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses. Comput. Educ. 104 , 18–33 (2017).

Kizilcec, R. F. et al. Scaling up behavioral science interventions in online education. Proc. Natl Acad. Sci. USA 117 , 14900–14905 (2020).

Flay, B. R. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev. Med. 15 , 451–474 (1986).

Rosqvist, J., Thomas, J. C., & Truax, P. Effectiveness versus efficacy studies. In J. C. Thomas & M. Hersen (Eds.), Understanding Research in Clinical and Counseling Psychology (pp. 319–354). New York, NY: Routledge (2011).

Bryk, A. S., Gomez, L. M., Grunow, A. & LeMahieu, P. G. Learning to improve: How America’s schools can get better at getting better . (Harvard Education Publishing, Cambridge, MA, 2015).

Chen, P. The strategic resource use intervention. In G. M. Waltion, & A. J. Crum (Eds.), Handbook of wise interventions: How social psychology can help people change (pp. 166–190). Guilford Press. (2020).

Gollwitzer, P. M. Implementation intentions: strong effects of simple plans. Am. Psychol. 54 , 493–503 (1999).

Huberth, M., Chen, P., Tritz, J. & McKay, T. A. Computer-tailored student support in introductory physics. PLoS ONE 10 , e0137001 (2015).

Article   PubMed   PubMed Central   CAS   Google Scholar  

Matz, R. et al. Analyzing the efficacy of ECoach in supporting gateway course success through tailored support. LAK21: 11th International Learning Analytics and Knowledge Conference , 216–225 (2021).

Bryan, C. J., Tipton, E. & Yeager, D. S. Behavioural science is unlikely to change the world without a heterogeneity revolution. Nat. Hum. Behav. 5 , 980–989 (2021).

Hill, C. J., Bloom, H. S., Black, A. R. & Lipsey, M. W. Empirical benchmarks for interpreting effect sizes in research. Child Dev. Perspect. 2 , 172–177 (2008).

Kraft, M. A., Blazar, D. & Hogan, D. The effect of teacher coaching on instruction and achievement: a meta-analysis of the causal evidence. Rev. Educ. Res. 88 , 547–588 (2018).

Silverman, S. L. From randomized controlled trials to observational studies. Am. J. Med. 122 , 114–120 (2009).

Victora, C. G., Habicht, J. & Bryce, J. Evidence-based public health: moving beyond randomized trials. Am. J. Public Health (1971) 94 , 400–405 (2004).

Wasserstein, R. L. & Lazar, N. A. The ASA statement on p-values: context, process, and purpose. Am. Stat. 70 , 129–133 (2016).

Cumming, G. The new statistics: why and how. Psychol. Sci. 25 , 7–29 (2014).

Goh, J. X., Hall, J. A. & Rosenthal, R. Mini meta‐analysis of your own studies: Some arguments on why and a primer on how. Soc. Personal. Psychol. Compass 10 , 535–549 (2016).

Borenstein, M., Hedges, L. V., Higgins, J. P. & Rothstein, H. R. A basic introduction to fixed‐effect and random‐effects models for meta‐analysis. Res. Synth. Methods 1 , 97–111 (2010).

Powers, J. T. et al. Changing environments by changing individuals: the emergent effects of psychological intervention. Psychol. Sci. 27 , 150–160 (2016).

Bierman, K. L. et al. The effects of a multiyear universal social–emotional learning program: the role of student and school characteristics. J. Consulting Clin. Psychol. 78 , 156–168 (2010).

Austin, P. C. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46 , 399–424 (2011).

Steel, P. The nature of procrastination: a meta-analytic and theoretical review of quintessential self-regulatory failure. Psychol. Bull. 133 , 65–94 (2007).

Carvalho, P. F., Sana, F. & Yan, V. X. Self-regulated spacing in a massive open online course is related to better learning. npj Sci. Learn. 5 , 2 (2020).

Dweck, C., What having a “growth mindset” actually means. Harvard Business Review . (2016). Available online at: https://hbr.org/2016/01/what-having-a-growth-mindset-actually-means .

Pelletier, G. N., Goegan, L. D., Chazan, D. J. & Daniels, L. M. Agreeing is not the same as accepting: exploring pre-service teachers’ growth mindsets. Can. J. N. Scholars Educ./Rev. Canadienne des. Jeunes Chercheures et. Chercheurs en. Éducation 11 , 59–69 (2020).

Walton, G. M., Crum, A. J. (Eds.). Handbook of wise interventions . (Guilford Press, New York, NY, 2020).

Yeager, D. S. & Dweck, C. S. What can be learned from growth mindset controversies? Am. Psychol. 75 , 1269–1284 (2020).

Binning, K. R. & Browman, A. S. Theoretical, ethical, and policy considerations for conducting social–psychological interventions to close educational achievement gaps. Soc. Issues Policy Rev. 14 , 182–216 (2020).

Chen, P., Powers, J. T., Katragadda, K. R., Cohen, G. L. & Dweck, C. S. A strategic mindset: an orientation toward strategic behavior during goal pursuit. Proc. Natl Acad. Sci. USA 117 , 14066–14072 (2020).

Keiser, H. N., Sackett, P. R., Kuncel, N. R. & Brothen, T. Why women perform better in college than admission scores would predict: exploring the roles of conscientiousness and course-taking patterns. J. Appl. Psychol. 101 , 569–581 (2016).

Liu, O. L., Rijmen, F., MacCann, C. & Roberts, R. The assessment of time management in middle-school students. Personal. Individ. Differences 47 , 174–179 (2009).

Misra, R. & McKean, M. College students' academic stress and its relation to their anxiety, time management, and leisure satisfaction. Am. J. Health Stud. 16 , 41–51 (2000).

Virtanen, P. & Nevgi, A. Disciplinary and gender differences among higher education students in self‐regulated learning strategies. Educ. Psychol. 30 , 323–347 (2010).

Steele, C. M. & Aronson, J. A threat in the air: how stereotypes shape intellectual identity and performance. Am. Psychol. 52 , 613–629 (1997).

Center for Academic Innovation. Ecoach . (2021) https://ai.umich.edu/software-applications/ecoach/ .

Chen, P., Ong, D. C., Ng, J. & Coppola, B. P. Explore, exploit, and prune in the classroom: strategic resource management behaviors predict performance. AERA Open 7 , 1–14 (2021).

Balduzzi, S., Rücker, G. & Schwarzer, G. How to perform a meta-analysis with R: a practical tutorial. Evid.-Based Ment. Health 22 , 153–160 (2019).

Ho, D. E., Imai, K., King, G. & Stuart, E. A. MatchIt: nonparametric preprocessing for parametric causal inference. J. Stat. Softw. 42 , 1–28 (2011).

Chen et al. From Controlled Trials to Classrooms (Data Repository). Open Science Framework (2022) https://doi.org/10.17605/OSF.IO/6QEJ7 .

Download references

Acknowledgements

The ECoach team at U-M is pleased to acknowledge support from NSF IUSE grant 1625397, the University of Michigan’s Third Century Initiative, a Next Generation Learning Challenge Wave I grant, and the Alfred P. Sloan Foundation through the SEISMIC Project. Patricia Chen’s work is supported by a Singapore National Research Foundation Fellowship NRF-NRFF11-2019-0007. We appreciate Gregory M. Walton’s feedback on an earlier version of this manuscript, and thank the instructors and students of the classes that have graciously participated in this study.

Author information

Authors and affiliations.

Department of Psychology, National University of Singapore, Singapore, Singapore

Patricia Chen & Daniel X. Y. Foo

Institute for Applied Learning Sciences and Educational Technology, National University of, Singapore, Singapore

Patricia Chen

Department of Information Systems and Analytics, National University of Singapore, Singapore, Singapore

Dennis W. H. Teo

The Center for Academic Innovation, University of Michigan, Ann Arbor, MI, USA

Holly A. Derry, Benjamin T. Hayward, Kyle W. Schulz & Caitlin Hayward

Departments of Physics and Astronomy, College of Literature, Science, and the Arts, and School of Education, University of Michigan, Ann Arbor, MI, USA

Timothy A. McKay

Department of Psychology, University of Texas at Austin, Texas, USA

Desmond C. Ong

You can also search for this author in PubMed   Google Scholar

Contributions

P.C. and D.C.O. conceived the study idea. P.C., H.A.D., and B.T.H. designed the Exam Playbook. H.A.D., B.T.H., and T.A.M. coordinated data collection. With input from P.C., D.W.H.T., D.X.Y.F., K.W.S., and D.C.O. organized, cleaned, and analyzed the data. C.H. and T.A.M. provided administrative leadership. P.C., D.W.H.T., D.X.Y.F., and D.C.O. wrote the paper with feedback from co-authors.

Corresponding authors

Correspondence to Patricia Chen or Desmond C. Ong .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, reporting summary, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and Permissions

About this article

Cite this article.

Chen, P., Teo, D.W.H., Foo, D.X.Y. et al. Real-world effectiveness of a social-psychological intervention translated from controlled trials to classrooms. npj Sci. Learn. 7 , 20 (2022). https://doi.org/10.1038/s41539-022-00135-w

Download citation

Received : 31 August 2021

Accepted : 30 May 2022

Published : 29 August 2022

DOI : https://doi.org/10.1038/s41539-022-00135-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

psychological intervention literature

IMAGES

  1. (PDF) Psychological First Aid: psychological intervention in disaster

    psychological intervention literature

  2. Brief Psychological Interventions in Clinical Practice : Ann Williamson : 9780470513064

    psychological intervention literature

  3. Psychological Interventions

    psychological intervention literature

  4. 26% Need Psychological Intervention

    psychological intervention literature

  5. Psychological Interventions : Mary Ballou : 9780275948511

    psychological intervention literature

  6. (PDF) Methods of Psychological Intervention

    psychological intervention literature

VIDEO

  1. Psychological disorders lecture 1

  2. Amherst, N.Y.

  3. 3 3 Basic Psychological Needs

  4. psychological problems

  5. Psychological research 2

  6. according to psychology

COMMENTS

  1. What Are the Two Types of Literature?

    The two types of literature are written and oral. Written literature includes novels and poetry. It also has subsections of prose, fiction, myths, novels and short stories. Oral literature includes folklore, ballads, myths and fables.

  2. What Are the Two General Types of Literature?

    The two most general types of literature are fiction and nonfiction. Fiction is literature created through the author’s imagination, while nonfiction is literature based on fact. Within these two categories, literature can be broken down in...

  3. What Are Psychological Needs?

    Psychological needs such as such as confidence, achievement and morality are on a high level of human needs, Psychology Today explains. The most basic needs are physiological, such as breathing, eating and sleeping. Mid-level needs are soci...

  4. The effectiveness of psychological intervention for depression

    The effectiveness of psychological intervention for depression, anxiety, and distress in prostate cancer: a systematic review of literature · Authors.

  5. Do We Report the Information That Is Necessary to Give Psychology

    A Scoping Review of the Psychological Intervention Literature 2000-2018. Perspect Psychol Sci. 2022 Jan;17(1):226-238. doi: 10.1177/

  6. Effectiveness of psychological interventions to improve quality of life

    Randomized controlled trial psychological interventions conducted ... SR procedures but without searching grey literature and with only one

  7. Psychological Interventions for Higher Education Students in Europe

    of this systematic literature review is to analyze psychological interventions for university students in Europe. The review was conducted

  8. Evidence-based psychological interventions in the treatment of

    In 2018 the APS completed the fourth edition of the Evidence-based psychological interventions in the treatment of mental disorders: A review of the literature.

  9. Evidence-based Psychological Interventions

    update the APS document Evidence-Based. Psychological Interventions in the Treatment of Mental. Disorders: A Literature Review (3rd edition). This review.

  10. Systematic literature review of psychological interventions for first

    We conducted a systematic review of the empirical literature examining the effectiveness of psychological interventions for post-traumatic

  11. Effectiveness of Mobile App-Based Psychological Interventions for

    The literature search was conducted in scientific databases, namely, ... addressing psychological interventions for college students.

  12. A review of the literature relating to psychological interventions and

    A review of the literature relating to psychological interventions and people with intellectual disabilities: Issues for research, policy, education and

  13. Brief Early Psychological Interventions Following Trauma

    A systematic literature search/review was undertaken of brief early psychological interventions following trauma. Only six randomized controlled trials wer.

  14. Real-world effectiveness of a social-psychological intervention

    Social-psychological interventions have raised the learning and ... We borrow these terms from the medical and public health literature