Psychometric validation of the Korean Patient-Reported Outcome Measurement Information System (PROMIS)-29 Profile V2.1 among patients with chronic pulmonary diseases

Introduction

Over 500 million people have chronic pulmonary disease (CPD) globally, imposing an immense worldwide health burden (1,2). CPD patients commonly experience multiple respiratory symptoms, including breathlessness, cough, dyspnea, and phlegm (3,4). In addition, patients experience periodic exacerbations, defined as an acute worsening of their CPD, resulting in unscheduled clinic or emergency-department visits and hospitalization for treatment (5,6). There have been many studies supporting that these respiratory symptoms and exacerbations have negative impact on quality of life among CPD patients (7-9). Furthermore, non-respiratory symptoms such as fatigue and anxiety themselves worsen the CPD patients’ quality of life, by limiting physical activities of daily living, and causing sleep disturbance at night-time (7,10-12).

Lung function, such as forced expiratory volume in 1 second and forced vital capacity, are often used to estimate the disease severity, however, they fail to capture the systemic manifestations and patient-experienced disease impact (13). Disparity between patient experience and physiological test results complicates patient care and underscores the importance of incorporating the patient’s perspective during counselling and treatment decisions (14). Patient-reported outcome (PRO) measures, defined as any report regarding the status of a patient’s health condition coming directly from the patient without interpretation of the patient’s response by a clinician or anyone else (15), provide the ideal means of systematically capturing the patient’s perspective and experience (16).

To measure PROs in CPD patients, the chronic obstructive pulmonary disease (COPD) assessment test (CAT), modified Medical Research Council (mMRC) dyspnoea scale, and St. George's Respiratory Questionnaire have commonly been used to assess respiratory symptoms, such as cough, sputum production, chest tightness, and dyspnoea (13). However, the CAT or mMRC were limited to evaluate worsening of extra-pulmonary symptoms such as fatigue, depression, anxiety, and reduced social functioning (17-19). In fact, generic measurements, such as the EuroQoL-5D (EQ-5D) (20) and the 36-item short-form health survey (SF-36) (21) were frequently used to assess the health-related quality-of-life (HRQoL) in CPD patients (22,23). However, the EQ-5D had relatively larger ceiling effect than other HRQoL measures and do not discriminate well severe and mild CPD patients (24). While SF-36 is more comprehensive than the EQ-5D (25), it still does not cover important symptoms for patients with CPD such as fatigue and anxiety (26). In 2010, the National Institutes of Health in the U.S. developed the Patient-Reported Outcomes Measurement Information System (PROMIS)-29 is a multi-item measure for assessing generic profile HRQoL (27). The PROMIS-29 covers frequently reported symptoms such as fatigue and sleep disturbance in CPD patients which are not covered by other generic PRO measures. The instrument has been used to assess HRQoL in patient with chronic disease and older adults with multiple chronic conditions including arthritis, cancer, congestive heart failure, diabetes, osteoporosis, and stroke (28). Thus, we aim to examine the validity of the PROMIS-29 among CPD patients. Thus, we aim to examine the validity of the Korean PROMIS-29 (K-PROMIS-29) among CPD patients.

We present the following article in accordance with the STROBE reporting checklist (available at https://dx.doi.org/10.21037/jtd-21-591).

Methods

Study participants and procedure

From September to October 2018, we conducted a cross-sectional survey at respiratory disease outpatient clinics at Samsung Medical Center in Seoul, South Korea to evaluate the HRQoL among patients with respiratory disease. Of total 304 patients who were aged ≥18 years and able to speak and read Korean participated in the survey. To validate the K-PROMIS-29 among CPD patients, we included 212 patients with pulmonary tuberculosis, nontuberculous mycobacterial lung disease, bronchiectasis, COPD, lung cancer, interstitial lung disease, and asthma in this study. We excluded patients who had history of severe cognitive impairment or Alzheimer diseases according to electronic medical records (EMRs). Two researchers of the study team explained the survey purpose and procedures to the participants. After providing informed consent, participants were requested to complete the questionnaire manually. This study was approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB number: SMC-2017-03-103-012). The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Measurement

We used the Korean version of the PROMIS-29 Profile V2.1 (K-PROMIS-29 V2.1) obtained from the PROMIS Health Organization (29). The K-PROMIS-29 V2.1 was translated into Korean using the Functional Assessment of Chronic Illness Therapy (FACIT) translation methodology (30). The PROMIS-29 V2.1 comprises 29 items in the following 7 domains: physical function, anxiety, depression, fatigue, sleep disturbance, ability to participate in social roles and activities, pain interference, and pain intensity. We used a 5-point Likert scale (range, 1–5) to measure symptom severity or frequency. The single pain intensity item was scored separately, and the response scale ranged from 0 (no pain) to 10 (worst pain imaginable). Questions regarding physical function and ability to participate in social roles and activities did not provide a specific time frame. Regarding the other 5 domains, questions were asked concerning the past 7 days. Domain scores were obtained by summing the item scores for each domain. The range of each domain was 4 to 20. Higher scores represent better physical function, ability to participate in social roles and activities, and more severe levels of anxiety, depression, fatigue, sleep disturbance, pain interference and pain intensity. In addition, we also converted the raw score into t-scores standardized for the general U.S. population {mean [SD] 50 [10]} using a T-score metric via Assessment Center (https://www.assessmentcenter.net; Northwestern University, Evanston, Illinois, USA) (27,31).

To examine convergent and discriminant validity, we used the Short Form Health Survey version-2.0 (SF-36v2), which is the most widely used tool for measuring generic health status with a 4-week recall period and has been well established in the Korean language (32-34). The SF-36v2 comprises the following 36 items in 8 domains: physical functioning, role limitations due to physical functioning, bodily pain, general health perceptions, vitality, social functioning, role limitations due to emotional functioning, and mental health. All items were rated on a Likert-type or frequency response scale, ranging from 3 response categories for physical functioning items to 6 response categories for bodily pain items. Using the standard scoring algorithm, scale scores were linearly transformed to range from 0 to 100, with higher scores representing superior health status (35). In this study, the Cronbach alpha of SF-36 was 0.96.

In CPD patients at the study institution, CAT scores and mMRC dyspnoea are routinely assessed at outpatient clinics; hence, we use these scores from EMRs which was administered to the patients on the same day patients participated in the survey. The CAT comprises the following 8 items: cough, phlegm, chest tightness, breathlessness, limited activities, confidence leaving home, sleeplessness, and energy, defined using contrasting adjectives. Each item was presented as a semantic 6-point (0–5) differential scale, providing a total score ranging from 0 to 40 points (36). The Cronbach alpha of CAT in this study was 0.91. The mMRC scale is a 5-point scale (0–4) based on dyspnoea severity (37). The mMRC had a moderate and strong correlation with breathless walking upstairs (r=0.53) and with home activity limited (r=0.69) in CAT score, respectively.

Furthermore, we asked study participants questions regarding their sociodemographic characteristics, including marital status, education level, monthly family income, and working status. Clinical characteristics were obtained from EMRs.

Statistical analyses

Analyses were conducted using raw scores. To assess K-PROMIS-29 V2.1 reliability, we calculated each domain’s internal consistency using Cronbach’s α and each item’s item-rest correlation. It is generally accepted that an α value of 0.6–0.7 indicates an acceptable level and that of ≥0.8 a very good level of reliability (38).

An exploratory and confirmatory factor analysis (CFA) was performed to test structural validity of the K-PROMIS-29 V2.1. For the exploratory factor analysis (EFA), a common factor model with an alpha factor extraction was used (39). With the CFA, we used the maximum likelihood to test whether our factor structure fit the data. Several goodness-of-fit indices were used to evaluate the model fit, including the goodness-of-fit index (GFI), comparative fit index (CFI), and standardised root mean squared residual (SRMR). A GFI and CFI >0.9 and SRMR <0.08 indicate a good fit to the data (40).

To examine convergent and discriminant validity, hypotheses on the direction and magnitude of Pearson’s correlations between the K-PROMIS-29 V2.1 and SF-36v2 were formulated a priori (41). We expected high correlations between conceptually similar domains in K-PROMIS-29 V2.1 and SF-36v2 as convergent validity, and relatively low correlations between conceptually different domains as discriminant validity. Moderate (0.5 <|r| <0.7) or large correlations (|r| ≥0.7) were considered to indicate construct validity (42). Among the patients who responded to the CAT and mMRC, we also calculated Pearson’s correlations of the K-PROMIS-29 V2.1 with CAT and mMRC dyspnoea scores. We completed a pairwise deletion in the analysis.

All significance tests were two-tailed, and statistical significance was set at P<0.05. All data analyses were performed using STATA version 15 (StataCorp LLC, College Station, TX, USA).

Results

Study participants

A total of 212 participants were enrolled in the study, and 188 (88.7%) completed the study questionnaire. Among the 24 patients excluded from the study due to missing PROMIS-29 items, 17 (70.8%), 5 (20.8%), 1 (4.2%), and 1 (4.2%) had not answered 1, 2, 3, and >4 items, respectively. The most frequently unanswered question was that regarding sleep quality (n=10, 4.7%).

The mean age (SD) was 62.8 (12.0) years, and 19.2% had less than middle-school education (Table 1). Disease types included bronchiectasis (n=46, 24.5%), COPD (n=45, 23.9%), nontuberculous mycobacterial lung disease (n=25, 13.3%), interstitial lung disease (n=22, 11.7%), and others (n=50, 26.6%).

Internal consistency

Cronbach’s alpha coefficients of the 7 sub-domains in the K-PROMIS-29 V2.1 ranged from 0.77 to 0.95, indicating satisfactory internal consistency. Pain interference and ability to participate in social roles and activities had the highest Cronbach’s α coefficient (0.95). Item-rest correlations, when any one of the items was removed, varied from 0.46 to 0.92. While all the items had generally acceptable levels of item-rest correlation (≥0.60), the items “In the past 7 days, my sleep was refreshing” (r=0.46) and “In the past 7 days, I had difficulty falling asleep” (r=0.58) had relatively low correlations with other items related to sleep disturbance (Table 2).

Structural validity

In EFA, the factor loadings for the 6 retained and varimax rotated factors were obtained (Table 3). The variance explained by the 6-factor solution was 82%. While other domains confirmed our hypothesis regarding the original K-PROMIS-29 V2.1 constructs, “anxiety and fatigue” constituted 1 domain and sleep-disturbance items were separated as “sleep quality” and “sleep was refreshing”, among others.

In CFA, the K-PROMIS-29 V2.1 goodness-of-fit indices (Figure 1) were high (CFI =0.90, SRMR =0.06). However, the “problem with my sleep” and “difficulty falling asleep” items in the sleep domain had relatively large error rates. Regarding correlations between K-PROMIS-29 V2.1 domains, “depression and anxiety” (0.75) and “physical and social” (0.76) also showed strong correlation. However, the factorial correlation between sleep disturbance and other domains was relatively weak.

Figure 1 Confirmatory factor analysis of the K-PROMIS-29 V2.1 items. K-PROMIS-29 V2.1, Korean version of the Patient-Reported Outcome Measurement Information System 29 Profile V2.1.

Convergent validity

Regarding convergent and discriminant validity, fatigue in the K-PROMIS-29 V2.1 and vitality in the SF-36v2 demonstrated a strong correlation (r=−0.75). In addition, the correlations between physical function in the K-PROMIS-29 V2.1 and physical functioning (r=0.70) in the SF-36v2 were moderate. Regarding the ability to participate in social roles and activities in the K-PROMIS-29 V2.1, moderate correlations with role-physical (r=0.64), role-emotional (r=0.64), and social functioning (r=0.65) in the SF-36v2 were observed. However, sleep disturbance was weakly correlated with all SF-36v2 subdomains (Table 4).

Among CPD patients (n=81), CAT activity was moderately correlated with physical function (r=−0.65) and ability to participate in social roles and activities (r=−0.63) in the K-PROMIS-29 V2.1. Fatigue in the K-PROMIS-29 V2.1 and energy in the CAT were also moderately correlated (r=0.70). Furthermore, there was a moderate correlation (r=0.62) between sleep disturbance in the K-PROMIS-29 V2.1 and sleep in the CAT. Regarding correlations between the mMRC dyspnoea score and K-PROMIS-29 V2.1, the mMRC dyspnoea score was strongly correlated with physical function (r=−0.80) and moderately correlated with anxiety (r=0.67), depression (r=0.59), pain interference (r=0.69), and fatigue (r=0.70) (supporting Table S1).

Discussion

In this study, the K-PROMIS-29 V2.1 was found to be a reliable and valid measure of quality of life among CPD patients. The goodness-of-fit indices for the original K-PROMIS-29 V2.1 domains were high. Convergent PROMIS-29 V2.1 validity was demonstrated by its varying degrees of correlation with the SF-36v2.

In total, 88.7% of the participants responded to all the questions, signifying a higher completion rate than that reported in other studies (43,44). Considering that >30.9% of the study participants were aged >70 years and 19.2% had less than a middle-school education, the K-PROMIS-29 V2.1 appears to be a feasible instrument for evaluating HRQoL, regardless of age and literacy. In our study, the most commonly unanswered question was that regarding sleep quality (n=10, 4.7%). Participants might have missed this item because the question was formatted differently from others. The other questions were complete statements or questions that participants responded to using a Likert scale (for example, “I feel fatigued” response options: “Not at all”, “A little bit”, … “very much”). However, the sleep-quality question was an open-ended question posed as follows: “My sleep quality was…,” to which participants were obliged to choose the response that best described their sleep quality (very poor, poor, fair, good, or very good). In fact, in a previous study conducted in a Dutch population (45), >90% of the study participants marked “My sleep quality was …” as being one of the most difficult items to answer. The authors hypothesised that the item might have been difficult to understand because of the response options (45).

The results indicated that the measure’s internal consistency was high.

The instrument has been validated in patient with cancer (46), kidney disease (47) and chronic musculoskeletal pain (48). In this study, we also found the Cronbach’s α coefficients for all subdomains fell within the range of acceptable internal consistency (49). The CFA also confirmed our hypothesis regarding the original K-PROMIS-29 V2.1 constructs, except in the sleep disturbance subdomain. In this study, the item regarding “sleep refreshing” had relatively weak item-rest correlations (0.46) with the other items: “In the past 7 days, I had a problem with my sleep” and “In the past 7 days, I had difficulty falling asleep”. These items also had a large margin of error in CFA. In fact, they were related to different factors in EFA. In a previous study, sleep initiation and sleep continuity appeared as separate constructs, and people perceived “feeling refreshed in the morning” and “good sleep continuity” as good sleep (50). Similarly, our study participants perceived questions regarding “a problem with sleep” and “difficulty with falling asleep” as questions concerning “sleep initiation” and questions regarding “sleep quality” and “refreshment of sleep” as questions concerning “sleep quality”, which is strongly related to sleep continuity (50). Furthermore, the correlation between anxiety and fatigue was 0.75, which was a relatively strong association in CFA, and they were combined as 1 domain in EFA. In a previous study, anxiety and depressive disorders commonly cited 25–40% overlap (51). However, despite a set of common features, anxiety and depression are clearly not identical emotional states (52). Additional cognitive interviews are needed to confirm the patients’ thoughts.

Convergent K-PROMIS-29 V2.1 validity was demonstrated by its varying degrees of correlation with the SF-36v2. K-PROMIS-29 V2.1 domains correlated with their corresponding SF-36v2 subdomains, except for sleep, for which no comparable SF-36v2 element was applicable. Among CPD patients, while mMRC dyspnoea scores had high-to-moderate correlations with all K-PROMIS-29 V2.1 subdomains, the tool was evaluated using only one question regarding the degree of dyspnoea. It did not address other important CPD symptoms, such as coughing, sputum production, chest tightness, and depression. Nevertheless, CAT would be more effective in multidimensional assessments for respiratory symptom severity in CPD patients (36); however, anxiety, depression, and pain, which are frequent symptoms reported by CPD patients, were not included (53).

Recently, the respiratory disease field has progressed further in assessing patients’ daily living activities; however, there is limited research regarding HRQoL among CPD patients (54). Clinicians might hesitate to use HRQoL measures such as the PROMIS-29 due to several reasons. They might not use it because they do not have enough resources such as time and personal (55). In addition, researchers and clinicians would not use the PROMIS-29 because it is not a disease-specific measure and they do not know how to interpret it due to lack of data (56). While the PROMIS-29 instrument has been tested in other chronic disease patients but is not yet widely used among CPD patients in the clinical settings. More data are necessary to understand how the K-PROMIS-29 would be summarized and presented to patients diagnosed with CPD. This study had some limitations. First, we exclusively recruited individuals who visited a respiratory clinic at one institution in Korea; hence, these findings may not be generalisable to patients in other settings. However, we tested validity in participants who had low literacy levels, including approximately 20% of participants with very little education according to the FACIT methodology guidelines (30). Considering the characteristics of our study participants, the K-PROMIS-29 V2.1 has acceptable measurement properties for use in patients with diverse backgrounds and CPDs. Second, the study did not include an existing questionnaire that effectively measures sleep disturbance to confirm the convergent validity of the sleep disturbance subdomain in the PROMIS-29 V2.1. However, we confirmed that the K-PROMIS-29 V2.1 had acceptable convergent validity with sleep items in the CAT, which is a disease-specific measure for COPD.

In conclusion, this study provides psychometric evidence for the reliability and construct validity of K-PROMIS-29 V2.1 in a CPD population. Considering that the PROMIS-29 is one of the standard PRO measures recommended for initial outcome assessment (57), it is encouraged to use the K-PROMIS-29 both for research and clinical care of CPD patients.

Acknowledgments

Funding: Danbee Kang and Juhee Cho received grants from the Ministry of Food and Drug Safety (18182MFDS407) and Samsung Medical Center Research and Development Grant (SMX1210381) in 2018 and Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1I1A2074210).

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://dx.doi.org/10.21037/jtd-21-591

Data Sharing Statement: Available at https://dx.doi.org/10.21037/jtd-21-591

Peer Review File: Available at http://dx.doi.org/10.21037/jtd-21-591

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://dx.doi.org/10.21037/jtd-21-591). DK received grants from the Ministry of Food and Drug Safety (18182MFDS407) in 2018 and Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1I1A2074210). JC received grants from the Ministry of Food and Drug Safety (18182MFDS407) in 2018 and Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1I1A2074210). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Review Board (IRB) of Samsung Medical Center (IRB number: SMC-2017-03-103-012), and informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. GBD Chronic Respiratory Disease Collaborators. Prevalence and attributable health burden of chronic respiratory diseases, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet Respir Med 2020;8:585-96. [Crossref] [PubMed]
2. . GBD 2015 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life-years (DALYs) for 315 diseases and injuries and healthy life expectancy (HALE), 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet 2016;388:1603-58. [Crossref] [PubMed]
3. Walke LM, Byers AL, Tinetti ME, et al. Range and severity of symptoms over time among older adults with chronic obstructive pulmonary disease and heart failure. Arch Intern Med 2007;167:2503-8. [Crossref] [PubMed]
4. Kessler R, Partridge MR, Miravitlles M, et al. Symptom variability in patients with severe COPD: a pan-European cross-sectional study. Eur Respir J 2011;37:264-72. [Crossref] [PubMed]
5. Seemungal TA, Donaldson GC, Bhowmik A, et al. Time course and recovery of exacerbations in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2000;161:1608-13. [Crossref] [PubMed]
6. Wedzicha JA, Donaldson GC. Exacerbations of chronic obstructive pulmonary disease. Respir Care 2003;48:1204-13; discussion 1213-5. [PubMed]
7. Tsiligianni I, Kocks J, Tzanakis N, et al. Factors that influence disease-specific quality of life or health status in patients with COPD: a review and meta-analysis of Pearson correlations. Prim Care Respir J 2011;20:257-68. [Crossref] [PubMed]
8. Miravitlles M, Anzueto A, Legnani D, et al. Patient's perception of exacerbations of COPD--the PERCEIVE study. Respir Med 2007;101:453-60. [Crossref] [PubMed]
9. Chang JA, Curtis JR, Patrick DL, et al. Assessment of health-related quality of life in patients with interstitial lung disease. Chest 1999;116:1175-82. [Crossref] [PubMed]
10. Arpinelli F, Carone M, Riccardo G, et al. Health-related quality of life measurement in asthma and chronic obstructive pulmonary disease: review of the 2009-2014 literature. Multidiscip Respir Med 2016;11:5. [Crossref] [PubMed]
11. Machado A, Marques A, Burtin C. Extra-pulmonary manifestations of COPD and the role of pulmonary rehabilitation: a symptom-centered approach. Expert Rev Respir Med 2021;15:131-42. [Crossref] [PubMed]
12. Olveira C, Olveira G, Gaspar I, et al. Depression and anxiety symptoms in bronchiectasis: associations with health-related quality of life. Qual Life Res 2013;22:597-605. [Crossref] [PubMed]
13. Ekström M, Sundh J, Larsson K. Patient reported outcome measures in chronic obstructive pulmonary disease: Which to use? Expert Rev Respir Med 2016;10:351-62. [Crossref] [PubMed]
14. Lavorini F. Quantifying bronchodilator responses in chronic obstructive pulmonary disease trials. Br J Clin Pharmacol 2005;59:385-6. [Crossref] [PubMed]
15. U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research. U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006;4:79. [Crossref]
16. Deshpande PR, Rajan S, Sudeepthi BL, et al. Patient-reported outcomes: A new era in clinical research. Perspect Clin Res 2011;2:137-44. [Crossref] [PubMed]
17. Booth S, Johnson MJ. Improving the quality of life of people with advanced respiratory disease and severe breathlessness. Breathe (Sheff) 2019;15:198-215. [Crossref] [PubMed]
18. Blinderman CD, Homel P, Billings JA, et al. Symptom distress and quality of life in patients with advanced chronic obstructive pulmonary disease. J Pain Symptom Manage 2009;38:115-23. [Crossref] [PubMed]
19. Breslin E, van der Schans C, Breukink S, et al. Perception of fatigue and quality of life in patients with COPD. Chest 1998;114:958-64. [Crossref] [PubMed]
20. Brooks R. EuroQol: the current state of play. Health Policy 1996;37:53-72. [Crossref] [PubMed]
21. Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992;30:473-83. [Crossref] [PubMed]
22. Nolan CM, Longworth L, Lord J, et al. The EQ-5D-5L health status questionnaire in COPD: validity, responsiveness and minimum important difference. Thorax 2016;71:493-500. [Crossref] [PubMed]
23. Polley L, Yaman N, Heaney L, et al. Impact of cough across different chronic respiratory diseases: comparison of two cough-specific health-related quality of life questionnaires. Chest 2008;134:295-302. [Crossref] [PubMed]
24. Chen J, Wong CK, McGhee SM, et al. A comparison between the EQ-5D and the SF-6D in patients with chronic obstructive pulmonary disease (COPD). PLoS One 2014;9:e112389 [Crossref] [PubMed]
25. Rowen D, Brazier J, Roberts J. Mapping SF-36 onto the EQ-5D index: how reliable is the relationship? Health Qual Life Outcomes 2009;7:27. [Crossref] [PubMed]
26. Pickard AS, Yang Y, Lee TA. Comparison of health-related quality of life measures in chronic obstructive pulmonary disease. Health Qual Life Outcomes 2011;9:26. [Crossref] [PubMed]
27. Cella D, Riley W, Stone A, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol 2010;63:1179-94. [Crossref] [PubMed]
28. Rose AJ, Bayliss E, Huang W, et al. Evaluating the PROMIS-29 v2.0 for use among older adults with multiple chronic conditions. Qual Life Res 2018;27:2935-44. [Crossref] [PubMed]
29. Patient-Reported Outcomes Measurement Information System. [cited 2018 1 June]. Available online: https://www.healthmeasures.net/explore-measurement-systems/promis/obtain-administer-measures
30. Lent L, Hahn E, Eremenco S, et al. Using cross-cultural input to adapt the Functional Assessment of Chronic Illness Therapy (FACIT) scales. Acta Oncol 1999;38:695-702. [Crossref] [PubMed]
31. Assessment Center. [cited 2019 23 February]; Available online: https://www.assessmentcenter.net
32. Hyeon NB, Ug LS. Testing the Validity of the Korean SF-36 Health Survey. J Health Info Stat 2003;28:3-25.
33. Kim SH, Jo MW, Lee SI. Psychometric properties of the Korean short form-36 health survey version 2 for assessing the general population. Asian Nurs Res (Korean Soc Nurs Sci) 2013;7:61-6. [Crossref] [PubMed]
34. Han CW, Lee EJ, Iwaya T, et al. Development of the Korean version of Short-Form 36-Item Health Survey: health related QOL of healthy elderly people and elderly patients in Korea. Tohoku J Exp Med 2004;203:189-94. [Crossref] [PubMed]
35. Maruish M, Kosinski M, Bjorner J, et al. User's Manual for the SF-36v2 Health Survey. Quality Metric Incorporated, 2011.
36. Jones PW, Harding G, Berry P, et al. Development and first validation of the COPD Assessment Test. Eur Respir J 2009;34:648-54. [Crossref] [PubMed]
37. Bestall JC, Paul EA, Garrod R, et al. Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax 1999;54:581-6. [Crossref] [PubMed]
38. Ursachi G, Horodnic IA, Zait A. How Reliable are Measurement Scales? External Factors with Indirect Influence on Reliability Estimators. Procedia Economics and Finance 2015;20:679-86. [Crossref]
39. Ferketich S. Focus on psychometrics. Internal consistency estimates of reliability. Res Nurs Health 1990;13:437-40. [Crossref] [PubMed]
40. Hu Lt, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 1999;6:1-55. [Crossref]
41. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res 2010;19:539-49. [Crossref] [PubMed]
42. Hinkle DE, Wiersma W, Jurs SG. Applied Statistics for the Behavioral Sciences. 5th edition. Houghton Mifflin, 2003.
43. van Dis FW, Mols F, Vingerhoets AJ, et al. A validation study of the Dutch version of the Quality of Life-Cancer Survivor (QOL-CS) questionnaire in a group of prostate cancer survivors. Qual Life Res 2006;15:1607-12. [Crossref] [PubMed]
44. Fujimori M, Kobayakawa M, Nakaya N, et al. Psychometric properties of the Japanese version of the Quality Of Life-Cancer Survivors Instrument. Qual Life Res 2006;15:1633-8. [Crossref] [PubMed]
45. van Kooten JA, Terwee CB, Kaspers GJ, et al. Content validity of the Patient-Reported Outcomes Measurement Information System Sleep Disturbance and Sleep Related Impairment item banks in adolescents. Health Qual Life Outcomes 2016;14:92. [Crossref] [PubMed]
46. Kang D, Kim Y, Lim J, et al. Validation of the Korean Version of the Patient-Reported Outcomes Measurement Information System 29 Profile V2.1 among Cancer Survivors. Cancer Res Treat 2021; Epub ahead of print. [Crossref] [PubMed]
47. Tang E, Ekundayo O, Peipert JD, et al. Validation of the Patient-Reported Outcomes Measurement Information System (PROMIS)-57 and -29 item short forms among kidney transplant recipients. Qual Life Res 2019;28:815-27. [Crossref] [PubMed]
48. Deyo RA. Performance of a Patient Reported Outcomes Measurement Information System (PROMIS) Short Form in Older Adults with Chronic Musculoskeletal Pain. Pain Med 2016;17:314-24. [PubMed]
49. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34-42. [Crossref] [PubMed]
50. Libman E, Fichten C, Creti L, et al. Refreshing Sleep and Sleep Continuity Determine Perceived Sleep Quality. Sleep Disord 2016;2016:7170610 [Crossref] [PubMed]
51. Dobson KS. The relationship between anxiety and depression. Clinical Psychology Review 1985;5:307-24. [Crossref]
52. Eysenck MW, Fajkowska M. Anxiety and depression: toward overlapping and distinctive features. Cogn Emot 2018;32:1391-400. [Crossref] [PubMed]
53. Afroz N, Gutzwiller FS, Mackay AJ, et al. Patient-Reported Outcomes (PROs) in COPD Clinical Trials: Trends and Gaps. Int J Chron Obstruct Pulmon Dis 2020;15:1789-800. [Crossref] [PubMed]
54. Frei A, Williams K, Vetsch A, et al. A comprehensive systematic review of the development process of 104 patient-reported outcomes (PROs) for physical activity in chronically ill and elderly people. Health Qual Life Outcomes 2011;9:116. [Crossref] [PubMed]
55. Bendtsen P, Leijon M, Sofie Sommer A, et al. Measuring health-related quality of life in patients with chronic obstructive pulmonary disease in a routine hospital setting: feasibility and perceived value. Health Qual Life Outcomes 2003;1:5. [Crossref] [PubMed]
56. Broderick JE, DeWitt EM, Rothrock N, et al. Advances in Patient-Reported Outcomes: The NIH PROMIS(®) Measures. EGEMS (Wash DC) 2013;1:1015. [Crossref] [PubMed]
57. Paap MC, Bode C, Lenferink LI, et al. Identifying key domains of health-related quality of life for patients with chronic obstructive pulmonary disease: the patient perspective. Health Qual Life Outcomes 2014;12:106. [Crossref] [PubMed]