Buyer beware: understanding the assumptions behind health economic assessments in personalized cancer care

After ‘lies, damn lies and statistics’ (a quote widely attributed to the 19^th century British Prime Minister Benjamin Disraeli), health economics may represent the next most complex kind of analyses offered up to physicians in order to influence their practice. On the one hand, basic cost effectiveness calculations are straightforward. The extra benefit from the new approach (beyond that of the existing standard) divided by the extra cost for the new approach (beyond that of the existing standard) generates the incremental cost effectiveness ratio (ICER). The ICER is then normalized to present an incremental cost per unit of health, usually either the cost per life years gained or quality-adjusted life years gained (QALYs), and measured against a perceived acceptability threshold. However, the complexity and controversy of most health economics lies not in the calculation but in the assumptions made to generate the costs and the benefits used in these analyses in the first place.

In the 2014 paper by Djalalov et al., a Markov model was used to assess the cost effectiveness of finding and treating ALK positive lung cancer with crizotinib from the perspective of the Ontario Health Care System in Canada (1). To put this into context, the discovery and exploitation of ALK rearranged non-small cell lung cancer (NSCLC) arguably represents one of the key advances that has helped to shape our modern treatment philosophy in advanced lung cancer. When patients were preselected from the start for the presence of an ALK gene rearrangement, the benefit from crizotinib was remarkable. Treatment was routinely associated with objective response rates of 60-70%, median progression free-survival (PFS) times of 8-10 months and a very reasonable side effect profile (2,3). In the USA, the drug was heralded as a breakthrough and rapidly licensed on the basis of single arm trial results. Later phase III trials confirmed the superiority of the targeted approach in this molecularly defined population compared to both 1^st and 2^nd line standard chemotherapies (3,4).

Yet this breakthrough may not be implemented in some countries, simply because it is not considered cost effective. With multiple other examples of giving specific targeted drugs to specific molecular subtypes of disease occurring, it is becoming vitally important to accurately address the health economics of these personalized medicine scenarios. For if we don’t address the feasibility of actually delivering these breakthroughs to patients in the real world they will not be breakthroughs at all.

Early on, we had raised the idea that in the era of increasingly personalized cancer care, health economic assessments would also now have to take into account the cost of the biomarker screening in addition to the cost and benefit of the drug. When the predictive biomarker is either very cheap and/or present in a large proportion of the population, the cost per QALY is primarily influenced by the cost, benefit and side effects of the new drug in the target population; however the cost of the screening can become dominant at very low biomarker positivity rates, with the point of inflection occurring at higher frequencies as the screening test becomes more expensive (5). In the Canadian study, immunohistochemistry (IHC) for ALK was utilized as the primary screening assay (estimated at 40 Canadian dollars per patient) with FISH (estimated at 388 dollars/patient) reserved as a confirmatory assay in IHC positive cases. Benefit from comparator use (i.e., without crizotinib) and subsequent use (i.e., post-crizotinib) of 1^st line cisplatin-gemcitabine chemotherapy, 2^nd line pemetrexed chemotherapy and 3^rd line erlotinib was based on efficacy within a large, but molecularly unselected registry of Ontario patients from 2005-2009. The quality of life on crizotinib was estimated to be only 6% better than the quality of life on platinum doublet chemotherapy, and 18% worse than life on erlotinib.

In their analyses, the dominant contributor to the cost per QALY was the cost of the crizotinib. Crizotinib use cost an additional $255,970 per QALY including the screening costs and $250,632 per QALY assuming no costs from the screening. Both of these values would be above commonly considered acceptability thresholds in Canada. But what if we change some of the assumptions used in their model?

For example, we now know that crizotinib is, in general, very well tolerated. In addition, due to the dramatic effects of the crizotinib on the underlying ALK positive cancer, the quality of these patients’ lives is far better than other advanced disease patients even in the absence of treatment related side effects. Multiple news stories of ALK positive patients returning to normal lives, pursuing active sports, even running for public office and winning have been aired. What if we assume that the utility of time on crizotinib approaches perfect quality of life, say 90% of perfect quality of life?

In that circumstance, recalculating based on the methodology described in the Djalalov paper, the cost effectiveness ratio reduces dramatically from 255,970 per quality adjusted life year gained to $143,421 per quality adjusted life year gained. Although this still will not meet most accepted cost effectiveness thresholds, it demonstrates the sensitivity of their results to one of the key underlying assumptions.

The results may also be sensitive to the particular clinical treatment options selected. For example, what if pemetrexed was used as part of a platinum doublet in the 1^st line setting, as opposed to as monotherapy in the 2^nd line setting? While the Canadian study used cisplatin-gemcitabine as their assumed 1^st line platinum doublet, increasingly platinum-pemetrexed is the 1^st line doublet of choice in non-squamous patients in many countries, often with continuation maintenance of the pemetrexed after 4-6 cycles of the doublet have been completed (6,7). Platinum-pemetrexed is both more expensive than platinum-gemcitabine, but also potentially more efficacious in the ALK positive population. Within the Phase III 1007 study quoted by Djalalov, which compared crizotinib to either docetaxel or pemetrexed in the 2^nd line setting, while these two chemotherapies were perceived to be equivalent in terms of efficacy in an unselected population, this was clearly not the case in the ALK positive population (4,8). Several retrospective studies had already suggested that ALK positive patients may do particularly well with pemetrexed, which was then confirmed within the 2^nd line 1007 study with the pemetrexed arm having almost twice the PFS (crizotinib: 7.7 months, pemetrexed: 4.2 months, docetaxel: 2.6 months) and nearly four times the response rate of the docetaxel arm (29% vs. 7%) (3,9,10). Subsequent data from the 1^st line PROFILE 1014 Phase III study of crizotinib versus up to six cycles of platinum-p xemetrexed, which was not available at the time of the Djalalov modeling, showed that although crizotinib was again superior to the chemotherapy in the ALK positive population, the proportional increase in PFS in the 1^st line setting (55%; 7 vs. 10.9 months) was less than in the 2^nd line setting (83%) even without the use of continuation maintenance pemetrexed (4).

How these different factors would interplay in the ICER calculation is unclear without explicit modeling. Markov models, such as the one used by the Djalalov paper, first assume a beginning health condition (known as a ‘state’) and then estimate what would happen to a typical population, given what we know about likely health outcomes from treatment. The key to economic analyses of this type are ‘transition probabilities’ which represent the probability of transitioning from one health ‘state’ to another at a defined time point. The table in the Djalalov paper provides the ‘transition’ probabilities from one line of treatment to the next line of treatment (e.g., from crizotinib to platinum doublet), or to other outcomes (e.g., post-treatment but stable, to supportive care or to death). If the treatment line changes, this creates a cascade effect through the model. As pemetrexed is more expensive than gemcitabine, in isolation this would reduce the incremental cost of using crizotinib and the ICER of crizotinib would decrease. Yet, on the other hand, in isolation, the potential increased clinical effectiveness of the chemotherapy in the 1^st line setting would reduce the relative gain in efficacy from using crizotinib and the ICER would increase. The net effect would depend on many factors, ranging from the absolute costs and clinical effectiveness already mentioned to the transition probabilities and quality of life estimates used in each state.

With regard to the impact of molecular testing on the ICER, it should be recognized that IHC for ALK is still not nationally validated and no standardized test yet exists. Usually, if and when such an official test is developed, the cost of the assay will be significantly higher than a home-grown assay as described in the Canadian study. So what if we assume that the IHC test doesn’t cost $40 but actually $400 dollars? If, as assumed, 95% of eligible patients take the IHC test, this increases the cost per patient to $3,105 which increases the cost effectiveness ratio by around $24,000 to approximately $280,000 per quality adjusted life year gained when screening costs are included. This still shows that the dominant contributor to the ICER is the cost of the drug, but illustrates the impact of the screening assumptions in their model.

One of the key things to recognize in the Canadian model is that the incremental gain assessed for their ICER is minuscule as it is calculated on a population level—0.011 QALYs gained. Consequently, even relatively small changes in baseline assumptions can alter the cost effectiveness ratio dramatically. For example, doubling the frequency of the ALK fusion genes from 3.4% to 6.8% effectively halves the cost effectiveness ratio. We have previously modelled how using clinical enrichment strategies to determine who to test based on histology, smoking status and absence of other known driver oncogenes can dramatically increase the frequency of ALK positive disease in the tested population (for example, to over 30% in the most enriched scenario) while also reducing the absolute costs as the total population tested diminishes (5). On a medical basis, we have argued against this approach, as enrichment is not perfect and many ALK positive cases will be left behind who do not meet the classical picture of the disease. However, if a breakthrough treatment is potentially being denied to all patients in a health system because of the perceived health economics of an all-inclusive approach, accepting clinical enrichment as a necessary evil to allow at least some to benefit may be the most pragmatic approach in these difficult situations.

The purpose of all of the above remodeling of the Djalalov paper is not to argue whether crizotinib is or is not cost effective, but simply to illustrate that what seems like an unequivocal fact (cost per QALY of X or Y) is actually highly equivocal and subject to considerable local, national and international variation as the assumptions and costs within the analysis are customized to specific health system factors and emerging factors in relation to efficacy and tolerability. While the Djalalov paper is very balanced in its views, it has to be recognized that the data for standard therapeutic practices and the expert opinions used to generate information on the expected quality of life and transition probabilities of each state will, inevitably, have had specific viewpoints associated with them. Certainly, no one would deny that Ontario is part of the real-world, but is it any more representative of the whole real world than say, California, or Japan or France?

Finally, when assessing health economics, the impact of time also needs to be considered. ICERs often depend critically on data drawn from clinical trials in relation to medications that are on patent protection. The extent that the results of the trials reflect the later effectiveness in a broader population, or of the treatment being delivered by a more or less experienced group of physicians and the extent to which current prices reflect ultimate prices, will all influence how robust these ICERs will be over time. If, for example, screening becomes cheaper or a higher frequency of the abnormality becomes detectable due to advances in technology or understanding of the underlying biology; or treatment costs decline; or treatment effectiveness and tolerability increase as clinicians gain more experience with the new treatments, then ICERs will improve. The case of treatment of childhood acute lymphoblastic leukemia is instructive in this regard (11). Today, survival rates for childhood acute lymphoblastic leukemia have reached 80-90% and ICERs are in the range of $8,215 per quality adjusted life year gained—clearly meeting the usual cost effectiveness thresholds. Yet, many of the incremental advances in treatment over the past half century did not initially meet these thresholds.

So, while cost effectiveness is undeniably becoming an increasingly important hurdle to clear in order for modern breakthroughs to see the light of day, the consumers of these data should continue to recall the ancient principle of caveat emptor—buyer beware. To make sense of health economic data we must always strive to understand and at times question or update the assumptions about both cost and effectiveness that have been used in the calculations in order to determine their true applicability in whatever version of the real world we are living in at the time.

Acknowledgements

Disclosure: The authors declare no conflict of interest.

References

1. Djalalov S, Beca J, Hoch JS, et al. Cost effectiveness of EML4-ALK fusion testing and first-line crizotinib treatment for patients with advanced ALK-positive non-small-cell lung cancer. J Clin Oncol 2014;32:1012-9. [PubMed]
2. Camidge DR, Bang YJ, Kwak EL, et al. Activity and safety of crizotinib in patients with ALK-positive non-small-cell lung cancer: updated results from a phase 1 study. Lancet Oncol 2012;13:1011-9. [PubMed]
3. Shaw AT, Kim DW, Nakagawa K, et al. Crizotinib versus chemotherapy in advanced ALK-positive lung cancer. N Engl J Med 2013;368:2385-94. [PubMed]
4. Solomon BJ, Mok T, Kim DW, et al. First-line crizotinib versus chemotherapy in ALK-positive lung cancer. N Engl J Med 2014;371:2167-77. [PubMed]
5. Atherly AJ, Camidge DR. The cost-effectiveness of screening lung cancer patients for targeted drug sensitivity markers. Br J Cancer 2012;106:1100-6. [PubMed]
6. Scagliotti GV, Parikh P, von Pawel J, et al. Phase III study comparing cisplatin plus gemcitabine with cisplatin plus pemetrexed in chemotherapy-naive patients with advanced-stage non-small-cell lung cancer. J Clin Oncol 2008;26:3543-51. [PubMed]
7. Paz-Ares L, de Marinis F, Dediu M, et al. Maintenance therapy with pemetrexed plus best supportive care versus placebo plus best supportive care after induction therapy with pemetrexed plus cisplatin for advanced non-squamous non-small-cell lung cancer (PARAMOUNT): a double-blind, phase 3, randomised controlled trial. Lancet Oncol 2012;13:247-55. [PubMed]
8. Hanna N, Shepherd FA, Fossella FV, et al. Randomized phase III trial of pemetrexed versus docetaxel in patients with non-small-cell lung cancer previously treated with chemotherapy. J Clin Oncol 2004;22:1589-97. [PubMed]
9. Camidge DR, Kono SA, Lu X, et al. Anaplastic lymphoma kinase gene rearrangements in non-small cell lung cancer are associated with prolonged progression-free survival on pemetrexed. J Thorac Oncol 2011;6:774-80. [PubMed]
10. Lee JO, Kim TM, Lee SH, et al. Anaplastic lymphoma kinase translocation: a predictive biomarker of pemetrexed in patients with non-small cell lung cancer. J Thorac Oncol 2011;6:1474-80. [PubMed]
11. van Litsenburg RR, Uyl-de Groot CA, Raat H, et al. Cost-effectiveness of treatment of childhood acute lymphoblastic leukemia with chemotherapy only: the influence of new medication and diagnostic technology. Pediatr Blood Cancer 2011;57:1005-10. [PubMed]