Predictive models of malignant transudative pleural effusions
Introduction
The first step in the evaluation of patients with pleural effusion (PE) is to determine whether the effusion is a transudate (results from imbalances in hydrostatic and oncotic pressures, both capillaries, and the pleural space) or an exudate (secondary to a change in flows due to an increase in the permeability of the pleural capillaries or due to a block in lymphatic drainage of the pleura).
It is estimate that if the PE is a transudate, it is not usually necessary to perform any other procedure to establish the diagnosis, since to know the cause that produces it does not usually present with major difficulties (1). However, several studies have documented cases of malignant PE (MPE) that are biochemically compared as transudates (2-14) and for this reason it has been debated whether it is necessary to perform cytology routinely in all cases of pleural transudate.
Moreover, clinical guidelines have not established any protocol on biochemical parameters to determine routinely in pleural fluid (PF), which are limited to recommending some basic parameters that are amplified depending on the clinical suspicion in each patient (15,16). It is clear that the requirements of a first-level hospital are not the same as those of a reference tertiary on that has a specific Pleural Unit (17). In these, with an elevated patient volume and more resources, the study protocol of a PE can include the determination of a greater number of biochemical parameters such as, for example, N-terminal pro-brain natriuretic peptide, adenosine deaminase, or carcinoembryonic antigen (CEA), as in our case.
The objective of the study was to identify those PE transudates in which it would be justified to perform cytology in PF, using predictive models. The first includes only clinical-radiological characteristics and is intended for those centers with more limitations in performing more specific biochemical determinations in PF (Model 1). The second, also includes an analytical variable (tumor marker) and it is intended for hospitals with fewer restrictions in requesting biochemical tests in PF (Model 2).
Methods
We retrospectively reviewed the medical records of all patients with PE who arrived consecutively in the Pulmonology Department of a Tertiary Care Hospital, between 1 January 2009 and 30 November 2015. In all PE a diagnostic algorithm was applied (18). In patients with repeated thoracentesis, only the results of the first were considered.
The diagnosis of CHF, hepatic hydrothorax, nephrotic syndrome and trapped lung was established based on known criteria (19-22). PF was diagnosed as malignant if the cytology or pleural biopsy was positive for malignancy; paramalignant PEs were not taken into account (23). Other causes of PE were diagnosed based on predefined criteria (18).
The biochemical parameters used to distinguish between pleural exudates and transudates were Light et al. (24): PF/serum (S) protein ratio >0.5; PF/S LDH ratio >0.6 and PF LDH greater than two-thirds the upper limits of the laboratory’s normal serum LDH (320 IU/L in our case). Patients with suspected CHF were only punctured if they had unilateral PE, asymmetric bilateral PE, chest pain or fever (25). A citology of PF (with or without pleural biopsy) was performed in those cases in which the PE not decreased with medical treatment appropiate to the established diagnosis. The variables included in the analysis are shown in the online supplemental material (Appendix 1). The methodology to determine the various biochemical parameters, both PF and serum, is detailed in the Appendix 2.
It was considered images suggestive of malignancy the presence of lung nodules/masses, pulmonary atelectasis, or mediastinal lymph node disease (malignancy X-ray/CT) in the chest X-ray or thoracic CT.
Statistical analysis
Logistic regression analysis was used to estimate the probability of an MPE. Two prognostic models were considered. Model 1 only included the clinical and radiological variables. In Model 2, a combination of clinical and radiological variables plus CEA (10-logarithm transformed, Log10CEA) was entered. Beginning with a model containing all potential covariates, the variable with the least significant p value was removed and tested using the likelihood-ratio test until all variables left in the model significantly (at α=0.05) contributed to the model.
The different aspects of model performance of the regression models were then studied, including calibration, discrimination, and diagnostic classification accuracy (26,27). Calibration was assessed using the Brier score, the Hosmer-Lemeshow goodness-of-fit test, and by plotting the non-parametric estimate of the association between the observed frequencies and the predicted probabilities for MPE. The receiver operating characteristics (ROC) curves and the corresponding area under the ROC curve (AUC) were calculated to test for discrimination. To ascertain the value of both models, we used the theoretical relationship between the threshold probability of having cancer and the relative value of false-positive and false-negative results. For this purpose, the classification rule was defined as follows: each patient was classified as belonging to the cancer group for those predicted probabilities equal or higher than the selected threshold and otherwise negative. This meant that the subjects in whom the model predicted cancer and they did not have it, would be false negatives, whilst if the model does not predict cancer and they do have it, they would be false positives. To correct optimism bias, bootstrap techniques were used by calculating the corrected versions of the discrimination and misclassification measures. Finally, graphical calculation devices (nomograms) were developed using both prediction models.
Statistical analyses were carried out in R using the packages “MASS”, “rms”, and “pROC”. These packages are freely available at http://cran.r-project.org (28).
All patients signed the informed consent before any procedure was performed. The protocol was evaluated and approved by the Clinical Research Ethics Committee of Galicia (registry 2015/478).
Results
During the study period, 1,181 patients with PE were seen in our department [276 MPE (23.4%), 255 with diseases that cause transudative PE (21.6%), and 650 due to other entities that produce exudative PE (55%)]. A total of 281 PE behaved as transudates, of which 26 (9.3%) were MPE. Table 1 shows the etiology of transudative PEs, and the origin of 26 MPE that behaved as transudates, as well as the total MPE (276 patients).
Full table
Appendix 3 illustrates the characteristics of 26 patients with transudative MPE and the most relevant findings are discussed. Table 2 shows clinical, radiological and analytical characteristics of patients with transudative PE according to the two groups that were classified (MPE and no-MPE). Concerning clinical-radiological variables, note that MPE are significantly younger (P=0.037), have more frequently left PE (P=0.027), more accompanying pulmonary lesions, more X-ray/CT findings suggestive of malignancy (P=0.000 for both), less dyspnea (P=0.009), more general syndrome (P=0.026) and more serosanguinous appearance (P=0.000). As regards analytical variables, MPE have significantly higher percentage of lymphocytes (P=0.04), cholesterol (P=0.001), proteins (P=0.049), albumin (P=0.009), adenosine deaminase (P=0.000), interleukin-1 (IL-1) (P=0.036), soluble interleukin-2 receptor (P=0.000), C-reactive protein (P=0.004) and tumor markers in PF (the latter also in serum). On the other hand, N-terminal pro-brain natriuretic peptide levels in serum, but not in PF, were significantly lower (P=0.004).
Full table
The coefficients obtained in the regression analysis for the different models evaluated are set out in Table 3. The diagnostic performances of the clinical-radiological data and analytical data were evaluated using models 1 and 2, respectively. The variables finally selected were: PE side, malignancy X-ray/CT, dyspnea, serosanguinous appearance, and CEA. Both models showed a good calibration. The highest discrimination capacity for the diagnosis of MPEs was obtained with Model 2 [AUC =0.995; P=0.008, in comparison to Model 1 (AUC =0.973)]. The bootstrap corrected AUCs in both models (Model 1, AUC =0.965; Model 2, AUC =0.983) were slightly lower than the apparent AUCs, reflecting no optimism.
Full table
The ROC curves are shown in Figure 1, and in Figure 2 the calibration graphs correspond to both models studied. The agreement between predicted probabilities and observed frequencies was excellent except at lower predicted probabilities where the observed frequencies were slightly higher.
Figure 3 shows the nomograms corresponding to Models 1 (Figure 3A) and 2 (Figure 3B). The weights of each one of the variables selected can be observed in these Figures and were the following. Model 1: left side, 40 points; malignancy X-ray/CT, 100 points; absence of dyspnea, 27 points; serosanguinous appearance, 61 points. Model 2: left side, 14 points; malignancy X-ray/CT, 29 points; absence of dyspnea, 9 points; serosanguinous appearance, 20 points and Log10CEA, between 11 and 100 points, for values of this marker comprised between −1 and 7 ng/mL. The probability of an MPE for a determined score is shown in the lower part of the picture.
In the online supplemental material (Appendix 4) an Excel calculator is included to facilitate application by clinicians to calculate the probability of malignancy in transudative PE, for each of the two models.
Table 4 shows the errors that are made (false positives and false negatives) for different cut-off points of the probabilities obtained after applying each one of the models. In this population, no false negatives were found for predicted probabilities lower than 11% and 14% in Models 1 and 2, respectively. However, on applying bootstrapping techniques, for not to find false negative results in 95% of possible samples, we would be required to use cut-off points for predicted probabilities of 3% and 4%, respectively.
Full table
Discussion
Our study confirms that both predictive models have a high discriminative value to predict an MPE and, therefore, to determine when we should perform cytology on a transudative PE.
Previous studies base the decision to perform cytology in the percentage of MPE that behave as transudates, which has led to some authors do not recommend performing cytology routinely (7,29), others do (8,9,12) and some suggest an intuitive approach (11) (Table 5). From our point of view, this approach is erroneous since, to make this decision, it must be from a scenario that will be found in the clinic, that is, is the PE a transudate or not, and it is, to know the conditions in which (which in our series is 9.3%), and a cytology must be requested in order not to delay the diagnosis too much, or subject the patient unnecessary examinations.
Full table
One relevant aspect is to know that an MPE can behave as a transudate, when all these cases should theoretically be exudates. Possible explanations could be: (I) in an initial state the accumulation of fluid could be due an obstruction of the lymphatic drainage by the tumor more than a direct infiltration of the pleura (2). In this circumstance, as the fluid that enters the pleural space is an ultrafiltrate with low protein levels, at least several weeks would be needed so that the proteins that are accumulated may be >50% of the serum concentration (30); (II) the PE is accumulated because there is another cause capable of producing a transudative PE, and is responsible for this, since a tumor that affects the pleura does not necessarily have to produce a PE (7). In many of the cases described in the literature, this second cause has been able to be demonstrated (7,10,11,13) (Table 5); and (III) the neoplasm and any of the previously mentioned causes contribute to the development of the PE. For all these reasons, it is difficult to establish in which circumstances cytology of the PE has to be performed on a transudate that could be due to the various mechanisms involve in the accumulation of the PE. The absence of clinical, radiological, and analytical data that suggests one of the diseases that commonly cause a transudate, they would be advised to request it, since in these circumstances tumor infiltration of the pleura could be the mechanism involved in its production. If other diseases are implicated in the appearance of the PE, the decision is more difficult to make. A lack of response, or a partial response to the treatment of these conditions, could indicate it. Finally, a good response, cannot rule out the presence of a neoplasm either, since there may be pleural involvement without giving rise to a PE (14). It is likely that these difficulties could contribute in that, up until now, there are no firm recommendations on those situations in which cytology is indicated in a transudative PE.
For outcome prediction, we considered presence or absence of malignant etiology. In this context, the logistic regression model is the most widely used statistical technique for binary medical outcomes (26). When developing a prediction model, we want to quantify how good the predictions from the model are (model performance). For this purpose, we performed calibration, discrimination, and diagnostic accuracy. We also are interested in the validity of the predictions for new subjects. We performed bootstrap resampling to correct overfitting and quantify optimism. Our results showed no overfitting when estimating the discrimination indexes (26).
Model 1, intended to be applied in centers with limitations in performing specific tests in the PF, was constructed with the variables, laterality of PE, malignancy X-ray/CT, absence of dyspnea and serosanguinous appearance of PE, due to its greater predictive capacity in the univariate analysis. This model shows an AUC of 0.973 and the variable with a greater discriminatory capacity is malignancy X-ray/CT (Table 3). The selected variables are consistent with the usually described clinical characteristics for MPE and for PE secondary to CHF, 85.9% of the non-MPE (219/251) of our series. In the PE due to CHF, dyspnea is practically a constant symptom, whilst in left PE it is rare (31). On the other hand, the MPEs frequently have a serosanguinous appearance, and radiological images suggestive of malignancy are a predictive variable of MPE (32). Thus, the absence of dyspnea, left PE, radiological images suggestive of malignancy and a fluid of serosanguinous appearance, each one with their respective weighting, will be what will determine the greater probability that a transudative PE is malignant.
Model 2, intended to be applied in hospitals with less restrictions in requesting biochemical tests in PF, it is constructed with the same clinical-radiological variables of Model 1 with the addition of the analytical variable Log10CEA. CEA was chosen, because it is the most extensively studied (33) and also showed a greater predictive capacity in the univariate analysis [AUC: 0.887 (0.829, 0.946)] than the other biomarkers (data not shown). This model shows an AUC of 0.995, significantly better than the Model 1 (P=0.008).
One crucial aspect is to decide the cut-off point for the probability in which we will indicate performing cytology. This decision is determined in this case by two aspects: (I) error rate (false positives and false negatives, Table 4) that we are able to assume; and (II) the economic cost of cytology. As this is a cheap test, it is preferable to avoid false negatives at the expense of an increase in false positives, especially if were consider that in nowadays treatments to neoplasm are more available. For Model 1, and assuming that there is no over-optimism, a possible cut-off point for the probability would be 10%, as no false negative would be obtained, with 31 false positives. With Model 2, of higher performance, the cut-off point would be 14%, with only 11 false positives and no false negatives. With the aim of avoiding over-optimism bias, due to having evaluated the performance of the predictive models from our own sample, we have used bootstrapping techniques so that it may be applied to different samples. With this, we may establish, with a 95% confidence interval, that by applying these predictive models to new patients, we will not find false negative results. This requires lowering the cut-off points for the predicted probabilities to 3% and 4% in Models 1 and 2, respectively. The number of errors with these probabilities are 32 and 18 false positive results, respectively. This assumes that cytology should be performed on those patients, who, on adding the scores of each variable, obtain a score of 69 points, if we apply Model 1 (Figure 3A), or 92, if we use Model 2 (Figure 3B).
The main limitation of this study is the small number of patients included with a transudative MPE, although it should be taken into account that is the largest series published up until now. Another limitation is that both predictive models are constructed based on patients recruited in a single center. It will be necessary to validate these results with multicenter studies involving a higher number of patients, including geographic areas with higher prevalence of tuberculous than malignant pleural effusions (MPEs), although tuberculous pleural effusions are very rarely transudates (34)”.
In summary, two applied models have a high value to predict when a transudative PE may be of neoplastic origin, being superior to that of adding an analytical variable to the clinical-radiological variables. These results suggest that both models should be applied to all patients with transudative PE and cytology should be indicated in all patients with a probability of an MPE of 3%, if Model 1 is applied, or 4%, if Model 2 is applied.
Acknowledgements
None.
Footnote
Conflicts of Interest: The authors have no conflicts of interest to declare.
Ethical Statement: All patients signed the informed consent before any procedure was performed. The protocol was evaluated and approved by the Clinical Research Ethics Committee of Galicia (registry 2015/478).
Appendix 1 Variables included in the analysis
The following variables were included in the analysis: gender, age, smoking, duration of symptoms, laterality of PE, accompanying pulmonary lesions, X-ray/CT compatible with malignancy, size of PE, dyspnea, chest pain, fever, general syndrome, macroscopic appearance of PE, red cell count, percentage and total nucleated cell count, determinations of glucose, amilase, total proteins, albumin, adenosine deaminase, interleukin-1, interleukin-6 and interleukin-8, soluble interleukin-2 receptors, tumor necrosis factor-α, C-reactive protein, CEA, carbohydrate antigen 15-3 and 19-9, cancer antigen 125, neuron-specific enolase, cytokeratin 21-1, tissue polypeptide-specific antigen and N-terminal pro-brain natriuretic peptide in serum and PF, and albumin gradient.
Appendix 2 Methodology used to determine biochemical parameters in PF and blood
PF and serum samples were collected at the same time with the patient fasting for at least 12 hours. The PF samples were centrifuged at 1,500 ×g for 10 min and processed on the same day that they were extracted.
Total white blood cells and differential counts were performed on the PF and peripheral blood samples using the ADVIA 2120 Hematology analyzer (Siemens Healthcare Diagnostics, Tarrytown, NY, USA). The PF differential count was performed using a conventional optical microscope.
Levels of total proteins (g/dL), albumin (g/dL), LDH (IU/L), cholesterol (mg/dL) and amilase IU/L), in PF and serum, were determined following the standard methodology in our hospital (ADVIA 2400, Siemens Healthcare Diagnostics Inc.).
N-terminal pro-brain natriuretic peptide was determined by means of electrochemiluminescence immunoanalysis (Modular Analytics E170, Roche Diagnostics) in accordance with the protocol of the manufacturer. This test has an interassay variation coefficient of 1.2–2.6% and a detection range of 5–15,000 pg/mL.
Interleukin-8 (pg/mL), interleukin-1 (pg/mL) and tumor necrosis factor-α (pg/mL) were measured using a solid-phase, two-site chemiluminescent immunometric assay (IMMULITE/IMMULITE interleukin-8, IMMULITE/IMMULITE interleukin-1 and IMMULITE/IMMULITE tumor necrosis factor-α) on an IMMULITE 1000 Analyzer (Siemens Healthcare Diagnostics, Llanberis, Gwynedd, Wales). Intra-assay precision (two levels) was 3.8% and 4.1% for interleukin-8, 3.1% and 4.9% for interleukin-1 and 3.5% and 3.6% for tumor necrosis factor-α; between-assay precision was 5.5% and 7.7% for interleukin-8, 5.1% and 9.1% for interleukin-1 and 6.5% and 4.4% for tumor necrosis factor-α.
Interleukin-6 (pg/mL) and soluble interleukin-2 receptor (U/mL) were measured using a solid-phase, two-site chemiluminescent immunometric assay (IMMULITE/IMMULITE interleukin-6 and IMMULITE/IMMULITE soluble interleukin-2 receptor) on an IMMULITE 2000 systems (Siemens Healthcare Diagnostics, Llanberis, Gwynedd, Wales). Intraassay precision (two levels) was 4.5% and 3.8% for IL-6 and 2.8% and 2.9% for soluble interleukin-2 receptor; interassay precision (two levels) was 3.9% and 4.1% for IL-6 and 5.3% and 5.4% for soluble interleukin-2 receptor.
CEA (ng/mL), carbohydrate antigen 15-3 (U/mL), cancer antigen 125 (U/mL), carbohydrate antigen 19-9 (U/mL), and Cytokeratin fragment 21-1 (ng/mL) were determined using an electrochemiluminescence analyser (MODULAR ANALYTICS Cobas E-601, Roche Diagnostics; Mannheim, Germany).
Neuron-specific enolase (ng/mL) was determined using electrochemiluminescence analyser (MODULAR ANALYTICS Cobas E-601, Roche Diagnostics; Mannheim, Germany) and tissue polypeptide-specific antigen (U/L) by chemiluminescence on an IMMULITE 1000 (Siemens Healthcare Diagnostics Products Ltd; Gwynedd, United Kingdom).
Adenosine deaminase activity (U/L at 37 °C) was determined colorimetrically using the Galanti and Giusti method (35). The NH4+ released by deamination of adenosine added to the samples was quantified by incubation with phenol nitroprusside in an alkaline medium, followed by measurement of absorbance at 628 nm. The within-run precision of this method in our hands was evaluated using 30 replicate high adenosine deaminase samples and 30 replicate low ADA samples. The corresponding coefficients of variation were 2.24% for low adenosine deaminase samples (mean + SD: 22.93±0.5 U/L) and 2.02 for high adenosine deaminase samples (102.48±2.04 U/L). Between-run precision was evaluated using 17 pairs of duplicates and a coefficient of variation of 2.51% (37.29±0.94 U/L) was obtained (36).
Appendix 3
Table S1 shows the characteristics of patients with a transudative MPE. Cut off values of NT-proBNP to differentiate between effusions of cardiac origin and others was established as >1,409 and 748 pg/mL in PF and S, respectively (37). Cut off values for CEA, carbohydrate antigen 15-3, cancer antigen 125, carbohydrate antigen 19-9, and cytokeratin fragment 21–1 were 3.5 ng/mL, 27.5 U/mL, 7.5 U/mL and 33 ng/mL respectively, the ones we use in our clinical practice.
Full table
The median age was 74 years (range, 32–86), with similar number of men and women (14/12). Median symptom duration was 14.5 days (range, 1–208 days). PE was right-sided in 8 patients (30.8%), left-sided in 12 (46.2%) and bilateral in 6 (23%). Approximately two thirds (65.4%) of patients (17/26) had lung lesions and 88.5% (23/26) had radiographic images suggestive of malignancy (X-ray or CT). Nine patients (34.6%) met the criteria for CHF. NT-proBNP values in PF were high in 16 patients (61.5%; 9 patients meeting CHF criteria and 7 other patients). The number of patients with high levels of CEA, carbohydrate antigen 15–3, cancer antigen 125, carbohydrate antigen 19-9 and cytokeratin fragment 21-1 were 17 (sensitivity 65.4%), 9 (34.6%), 7 (26.9%) and 10 (38.5%), respectively.
Appendix 4
An Excel calculator was constructed for clinician use to determine the probability of a transudative PE being of malignant origin, for both models.
References
- Light RW, Macgregor MI, Luchsinger PC, et al. Pleural effusions: the diagnostic separation of transudates and exudates. Ann Intern Med 1972;77:507-13. [Crossref]
- Decker DA, Dines DE, Payne WS, et al. The significance of a cytologically negative pleural effusion in bronchogenic carcinoma. Chest 1978;74:640-2. [Crossref]
- Valdés L, Pose A, Suàrez J, et al. Cholesterol: a useful parameter for distinguishing between pleural exudates and transudates. Chest 1991;99:1097-102. [Crossref]
- Romero S, Candela A, Martín C, et al. Evaluation of different criteria for the separation of pleural transudates from exudates. Chest 1993;104:399-404. [Crossref]
- Vives M, Porcel JM, Vicente de Vera M, et al. A study of Light's criteria and possible modifications for distinguishing exudative from transudative pleural effusions. Chest 1996;109:1503-7. [Crossref]
- Moltyaner Y, Miletin MS, Grossman RF. Transudative pleural effusions: false reassurance against malignancy. Chest 2000;118:885. [Crossref]
- Assi Z, Caruso JL, Herndon J, et al. Cytologically proved malignant pleural effusions: distribution of transudates and exudates. Chest 1998;113:1302-4. [Crossref]
- Castro DJ, Nuevo GD, Perez-Rodriguez E. Cytologically proved malignant pleural effusions. Chest 1998;114:1798. [Crossref]
- Foresti V, Scolari N, Villa A. Positivity of pleural fluid cytologic examination in transudative pleural effusions. Chest 1998;114:1798-9. [Crossref]
- Ashchi M, Golish J, Eng P, et al. Transudative malignant pleural effusions: prevalence and mechanisms. South Med J 1998;91:23-6. [Crossref]
- Porcel JM, Alvarez M, Salud A, et al. Should a cytologic study be ordered in transudative pleural effusions? Chest 1999;116:1836-7. [Crossref]
- Moltyaner Y, Miletin MS, Grossman RF. Transudative pleural effusions: false reassurance against malignancy. Chest 2000;118:885. [Crossref]
- Ryu JS, Ryu ST, Kim YS, et al. What is the clinical significance of transudative malignant pleural effusion? Korean J Intern Med 2003;18:230-3. [Crossref]
- Gonlugur TE, Gonlugur U. Transudates in malignancy: still a role for pleural fluid. Ann Acad Med Singapore 2008;37:760-3.
- Villena Garrido V, Ferrer Sancho J, Hernández Blasco L, et al. Diagnosis and treatment of pleural effusion. Arch Bronconeumol 2006;42:349-72.
- Hooper C, Lee YC, Maskell N, et al. Investigation of a unilateral pleural effusion in adults: British Thoracic Society Pleural Disease Guideline 2010. Thorax 2010;65 Suppl 2:ii4-17. [Crossref]
- Bhatnagar R, Maskell N. Developing a 'pleural team' to run a reactive pleural service. Clin Med (Lond) 2013;13:452-6. [Crossref]
- Villena Garrido V, Cases Viedma E, Fernández Villar A, et al. Recommendations of diagnosis and treatment of pleural effusion. Update. Arch Bronconeumol 2014;50:235-49.
- Kapoor JR, Perazella MA. Diagnostic and therapeutic approach to acute decompensated heart failure. Am J Med 2007;120:121-7. [Crossref]
- Xiol X, Guardiola J. Hepatic hydrothorax. Curr Opin Pulm Med 1998;4:239-42. [Crossref]
- Llach F, Arieff AI, Massry SG. Renal vein thrombosis and nephrotic syndrome. A prospective study of 36 adult patients. Ann Intern Med 1975;83:8-14. [Crossref]
- Pereyra MF, Ferreiro L, Valdés L. Unexpandable lung. Arch Bronconeumol 2013;49:63-9.
- Sahn SA. Malignant pleural effusions. Semin Respir Med 1987;9:43-53. [Crossref]
- Light RW. Clinical practice. Pleural effusion. N Engl J Med 2002;346:1971-7. [Crossref]
- Light RW. Pleural effusions. Med Clin North Am 2011;95:1055-70. [Crossref]
- Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York:Springer. 2001.
- Ewout S. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009.
- R Development Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0.[Accessed 4 Mar 2015]. Available online: http://www.R-project.org
- Teklu B. Cytology on transudative pleural effusions. Chest 1999;116:846-7. [Crossref]
- Sahn SA. Malignancy metastatic to the pleura. Clin Chest Med 1998;19:351-61. [Crossref]
- Kataoka H. Pericardial and pleural effusions in decompensated chronic heart failure. Am Heart J 2000;139:918-23. [Crossref]
- Valdés L, San-José E, Ferreiro L, et al. Combining clinical and analytical parameters improves prediction of malignant pleural effusion. Lung 2013;191:633-43. [Crossref]
- Liang QL, Shi HZ, Qin XJ, et al. Diagnostic accuracy of tumour markers for malignant pleural effusion: a meta-analysis. Thorax 2008;63:35-41. [Crossref]
- Valdés L, Alvarez D, San José E, et al. Tuberculous pleurisy: a study of 254 patients. Arch Intern Med 1998;158:2017-21. [Crossref]
- Giusti G. Adenosine deaminase. In: Bergmeyer HU, ed. Methods of enzymatic analysis. New York: Academic Press 1974:1092-9.
- Valdés L, Alvarez D, San José E, et al. Value of adenosine deaminase in the diagnosis of tuberculous pleural effusions in young patients in a region of high prevalence of tuberculosis. Thorax 1995;50:600-3. [Crossref]
- Valdés L, José ES, Pose A, et al. Diagnostic value of N-terminal pro-brain natriuretic peptide in pleural effusions of cardiac origin. Arch Bronconeumol 2011;47:246-51.