COURT OF APPEAL (CIVIL DIVISION)
ON APPEAL FROM THE HIGH COURT OF JUSTICE
Mrs Justice Dobbs
Strand, London, WC2A 2LL
B e f o r e :
LORD JUSTICE JACOB
LORD JUSTICE RICHARDS
| Between :
The Queen (on the application of
|- and -
|National Institute for Health and Clinical Excellence
- and -
(1) Shire Pharmaceuticals Limited
(2) Association of the British Pharmaceutical Industry
Nigel Giffin QC and Daniel Stilitz (instructed by Beachcroft LLP) for the Defendant
Brian Kennelly (instructed by Ashurst LLP) for the First Intervener
Ivan Hare (instructed by the Association of the British Pharmaceutical Industry) made written submissions for the Second Intervener
Hearing dates : 14-15 April 2008
Crown Copyright ©
Lord Justice Richards :
NICE's system of technology appraisals
"12. When a particular topic has been selected for appraisal, NICE identifies those organisations which might have an interest in the outcome of the appraisal. These typically include groups representing patients and carers, bodies representing healthcare professionals, manufacturers, and research groups. Such stakeholders are divided into 'consultees' and 'commentators', with only consultees having a right of appeal against NICE's eventual Guidance. NICE works with the Department of Health (DoH) to produce a scope for the appraisal, which, once finalised, the DoH formally instructs NICE to carry out.
13. An independent academic centre is commissioned to review and evaluate evidence on the health technology under appraisal. It produces a Technology Assessment Report (TAR) which presents an analysis of the cost-effectiveness of the health technology. Often (as in the present case) the academic centre will build an economic model to inform its analysis. Consultees and commentators are invited to comment on the Assessment Report and to make submissions. The Assessment Report and comments made on it are then drawn together in an Evaluation Report.
14. The appraisal is then allocated to one of NICE's Appraisal Committees (AC). The Appraisal Committees are standing committees, the members of which are drawn from a range of backgrounds including eminent clinicians, health administrators, academics, representatives of the pharmaceutical industry and lay members.
15. The Appraisal Committee considers the Evaluation Report. It then hears evidence from clinical experts nominated by the consultees and commentators, and from patients and carers before making initial recommendations in an Appraisal Consultation Document (ACD) about the use of the health technology. Consultees and commentators have four weeks in which to comment on the ACD. The Appraisal Committee considers all comments received. It then makes its final recommendations in a Final Appraisal Determination (FAD).
16. The FAD is submitted to the NICE Guidance Executive for its approval. Once the FAD is approved, it is circulated to consultees and commentators. Consultees have 15 days during which to appeal against its recommendations. If there are no appeals, or if the appeals made are dismissed, the final recommendations are issued as NICE Guidance. If an appeal is successful, the Appraisal Committee reconsiders its recommendations.
17. The key principle underlying NICE's approach to appraisals is that the NHS's limited resources should be targeted on those treatments which provide best value for money. The principle is to be found at paragraph 4.1 of NICE's 'Social Value Judgments – principles for the development of NICE Guidelines' (8th December 2005) and is summarised by Professor Andrew Stevens, the chair of the Appraisal Committee which appraised the AChEIs in the present case:
'If money is spent on donepezil for patients with mild Alzheimer's disease, then that money is not available to spend elsewhere on the treatment of other patients. The role of NICE's appraisal committees is therefore to judge whether the clinical and cost effectiveness of a technology being appraised is such as to justify spending the NHS's money on that technology, rather than on one of the many other technologies competing for the finite resources available.'
18. In determining whether particular treatments are cost-effective, NICE generally seeks to ascertain the incremental cost per quality adjusted life year (or 'cost per QALY') of using a particular health technology. In broad terms, this is a standard measure of the cost of gaining a particular unit of utility through the use of the technology which can be applied across the whole spectrum of treatments appraised by NICE.
19. The general threshold for an 'acceptable' cost per QALY is approximately £20,000 per QALY: see 18.104.22.168 of NICE's 'Guide to the Methods of Technology Appraisal' (April 2004). Above approximately £30,000 per QALY, technologies are not normally recommended (para. 22.214.171.124), although there has been a small number of exceptions. Between £20,000 and £30,000 per QALY, NICE will consider whether there are specific considerations which justify recommending the technology, such as its innovative nature or the particular needs of the relevant patient group. The costs per QALY in this case were significantly in excess of these normal thresholds reflected even in the category for which NICE approved the use of AChEIs."
The appraisal process in this case
"20. … On 30th January 2004, consultees and commentators were sent the final scope for the appraisal of drugs for the treatment of AD. On 1st April 2004 consultees (including Eisai and the Interested Parties) were issued with the Protocol under which the cost-effectiveness of the AChEIs was to be assessed. On 3rd June 2004 written submissions were received from a total of twelve consultees and commentators, including Eisai and the Interested Parties. All four of the relevant manufacturers submitted their own economic models of the cost-effectiveness of the drugs under appraisal.
21. On 31st August 2004, NICE received the final Assessment Report from Southampton Health Technology Assessment Centre (SHTAC). This included an economic model to illustrate the cost-effectiveness of the AChEIs (the Model). The Model was in the form of an Excel spreadsheet. Both the design of and the assumptions underlying the Model were explained in Section 6 of the Assessment Report.
22. The initial assessment of the cost-effectiveness of Donepezil, based on the available clinical evidence, indicated a cost per QALY of approximately £94,000. This was far outside the range within which a treatment might be recommended as cost-effective. The view reached by the Appraisal Committee was, that if recommended for use amongst all mild to moderate AD sufferers, the AChEIs would not come even close to achieving the levels of cost-effectiveness generally required for recommendation by NICE for use within the NHS.
23. On 17th September 2004, SHTAC's Assessment Report was sent out for consultation. All consultees and commentators were offered a copy of the Model (in 'read-only' format). Eisai did not take up this offer. On 4th October 2004, comments on SHTAC's Assessment report were received from ten consultees and commentators, including Eisai and the Interested Parties.
24. On 20th October 2004, the Appraisal Committee held its first meeting to discuss the formulation of the ACD. The Appraisal Committee requested that further analysis be undertaken on the cost-effectiveness of the AChEIs by NICE's secretariat before an ACD could be published. This was in response to representations made by consultees. In particular, the Appraisal Committee specified that alternative assumptions, which were generally more favourable to the drugs than the original assumptions made, should be used in the Model. This new set of assumptions was referred to as the 'augmented base case'. In addition, the Appraisal Committee asked that sensitivity analyses be carried out in relation to certain assumptions in the augmented base case to see if the Model was particularly sensitive to these changes.
25. In accordance with the Appraisal Committee's request, NICE's technical staff carried out further analyses of the cost-effectiveness of the AChEIs based on a variety of alternative assumptions. They also carried out further sensitivity analyses of the Model. These analyses were set out in Technical Report No 1 (TR1), circulated on 7th January 2005 to all consultees and commentators for their observations. The effect of the augmented base case was to produce very similar conclusions on the benefits of treatment with AChEIs to those produced by the models submitted by the manufacturers. This further analysis indicated a cost per QALY for Donepezil of approximately £54,000. This figure was still well outside the range for which the treatment might be recommended for AD sufferers as a whole.
26. On 21st January 2005, comments on Technical Report No 1 were received from ten consultees and commentators, including Eisai and the Interested Parties. On 25th January 2005 the Appraisal Committee met for a second time to consider Technical Report No 1 and the comments received.
27. By an email dated 3rd February 2005, all consultees and commentators were again given the opportunity to receive a copy of the Model, now in revised form and including the augmented base case, in 'read-only' format. On 8th February 2005, Eisai, for the first time, requested a copy of the Model. On the same day, NICE sent Eisai a copy of the Model, in 'read-only' format.
28. Following the second meeting of 25th January 2005, the Appraisal Committee reached a preliminary view not to recommend the use of AChEIs for the treatment of AD at all. The drugs were not considered cost-effective enough to justify their use, bearing in mind their limited clinical effectiveness. This view was set out in the first Appraisal Consultation Document (ACD1). ACD1 was sent to all consultees and commentators on 21st February 2005, and further comments were invited. ACD1 was published on NICE's website on 1st March 2005.
29. On 28th February 2005, Eisai wrote to NICE requesting a fully-executable copy of the Model. On 4th March 2005 NICE replied to Eisai explaining its policy to limit disclosure to 'read-only' copies of its economic models. On 22nd March 2005, responses to ACD1 were received from eighteen consultees and commentators, including Eisai and the Interested Parties. In total, over 8,000 contributions were received during the consultation on ACD1. One recurring theme of this consultation was that, if a group of patients who did respond to the AChEIs could be identified, (a so-called 'responder group'), and treatment were targeted on that group, then the drugs might be sufficiently cost-effective at least for that group.
30. In response to this, NICE, on 12th April 2005, requested data from the consultees and commentators, as well as from practitioners and others with an interest in data collection relating to the use of AChEIs in clinical practice. In particular, NICE sought data on patients who had responded, and data on how long non-responders had remained on treatment with AChEIs. On 20th April 2005 Eisai responded to this request by providing additional data, as did other consultees and commentators.
31. On 1st June 2005, the Appraisal Committee met for the third time. It considered the additional data submitted in response to NICE's request. Additional economic modelling was carried out on the basis of some of the suggestions made by consultees. The Appraisal Committee was not, however, persuaded by the data provided, purporting to show that initial responders could be regarded as a sub-group of patients that enjoyed significantly better-than-average benefits from treatment with AChEIs. Following the meeting on 1st June 2005, the Appraisal Committee prepared a Final Appraisal Determination (FAD). However, uncertainty remained as to whether it might be possible to identify a sub-group of patients for whom the AChEIs could be demonstrated to be cost-effective. NICE's Guidance Executive reached the view that, before a FAD was issued, further analysis should be undertaken. Unusually, therefore, the FAD was not issued at this stage. Instead, a request was made by NICE to the manufacturers of the AChEIs (including Eisai) for further individual patient-level analyses of data from the key clinical trials. NICE held a meeting with the manufacturers on 10th August 2005 to ask for such data. Following submission of the requested data, NICE held a meeting with the manufacturers on 31st October 2005 and asked for still further analyses. On 6th November 2005 Eisai provided further analyses in response to this request.
32. The Medical Research Council's Biostatistics Unit was commissioned by NICE to interpret the data provided by the manufacturers and to report on it to the Appraisal Committee. NICE's technical staff then used the data report by the MRC Biostatistics Unit to re-run the augmented base case Model. The results of this re-running of the economic model were reported in Technical Report No 2 (TR2), issued on 25th November 2005. This report was provided to all consultees. Eisai did not ask for a copy of the re-run economic model (in non-executable form) at this stage, but did provide a response to the report on 7th December 2005. On 9th December 2005 a further twelve responses to Technical Report No 2 were received from commentators and consultees, including the Alzheimer's Society.
33. In the light of the additional analyses carried out, the Appraisal Committee now found that, by adjusting its assumptions in the drug's favour, and by limiting treatment to patients with moderately-severe (as opposed to mild) AD, the cost-effectiveness of the AChEIs began to approach the level usually required for a treatment recommended for use in the NHS. On this approach, the estimates of cost per QALY for donepezil ranged from £31,000 to £38,000. For the AChEIs, taken as a class of drugs, a cost per QALY of £35,000 was arrived at.
34. On 20th December 2005, the Appraisal Committee met for the fourth time. This led to the formulation of the second Appraisal Consultation Document (ACD2), which was sent to all consultees and commentators on 16th January 2006. It was published on NICE's website on 23rd January 2006. On 10th February 2006 Eisai submitted its response to ACD2. On 13th February 2006 comments on ACD2 were received from twenty further consultees and commentators, including the Interested Parties.
35. On 27th April 2006, the Appraisal Committee met for the fifth time to consider ACD2 and the comments that had been received on it. This led to the formulation of the Final Appraisal Determination (FAD). On 25th May 2006 the FAD was sent to all consultees and commentators. It was published on NICE's website on 26th May 2006 …."
The disclosure issue
"The Assessment Group may produce an economic model in support of the Assessment Report. If the model does not contain information that was designated as confidential in the submission, the Institute offers consultees and commentators the opportunity to receive by email a read-only version of the model, for information only. Requests for the model must be made in writing, and it is supplied on the basis that the consultee or commentator agrees, in writing, to the following conditions for its use.
- The economic model and its contents are confidential and are protected by intellectual property rights, which are owned by the relevant Assessment Group. It cannot be used for any purpose other than to inform the recipient's understanding of the Assessment Report.
- The model must not be re-run with alternative assumptions or inputs.
- The consultees or commentators will not publish the model wholly or in part, or use it to inform the development of other economic models."
The decisions of the Appeal Panel and of Dobbs J
"62. In summary, the Claimant had more than sufficient information to make intelligent response and to give proper advice. What was not possible was to 'run or play with the system' to see how it would react to different data and assumptions being put in. Eisai was not, however, deprived of the opportunity of putting forward suggestions and concerns in the light of what it was able to observe. So intelligent indeed were some of the responses and advice that alterations were made by NICE to the Model as a result. Whilst highlighting its inability to run the model and thereby test its robustness and accuracy, Eisai was nevertheless still able to advise on the need to check the accuracy of figures and to run sensitivity analyses with the different assumptions. I agree with the decision of the Appeal Panel that the withholding of the fully executable model was not unfair."
The legal principles
"108. … To be proper, consultation must be undertaken at a time when proposals are still at a formative stage; it must include sufficient reasons for particular proposals to allow those consulted to give intelligent consideration and an intelligent response; adequate time must be given for this purpose; and the product of consultation must be conscientiously taken into account when the ultimate decision is taken ….
112. … It has to be remembered that consultation is not litigation: the consulting authority is not required to publicise every submission it receives or (absent some statutory obligation) to disclose all its advice. Its obligation is to let those who have a potential interest in the subject matter know in clear terms what the proposal is and exactly why it is under positive consideration, telling them enough (which may be a good deal) to enable them to make an intelligent response. The obligation, although it may be quite onerous, goes no further than this."
"One cannot help feeling that the denial of the applicants' request was due to an inbuilt reluctance to give reasons or disclose advice lest it give opponents fuel for argument. One can understand and respect the need for ministers to preserve confidentiality as to the in-house advice they receive on administrative and political issues from their civil service staff. But here, the advice was from a body of independent experts set up to advise the Secretary of State on scientific matters I can see no ground in logic or reason for declining to show the applicants the text of the advice. In view of the total change of policy the Regulations would bring about and its unique impact on the applicants, fairness demanded that they should be treated with candour. To conceal from them the scientific advice which directly led to the ban was, in my judgment, unfair and unlawful."
Application to the facts
"18. As NICE itself recognizes when requesting that models are submitted for its review, it is not possible to properly assess a model that is locked and does not allow changes to be made to inputs. The reason is that one of the main ways of detecting problems in a model is to carry out 'extreme value analysis' where input values are chosen so that the analyst can predict the results. For example, [if] is the probability of death is set to zero, then no deaths are expected and this can be checked easily. In this way, the technical validity of the model can be ascertained.
19. In addition, assessing a model requires elucidating its structural connections and the formulas that implement these. Although it is possible to view the formulas, it is difficult to follow them in a locked model as none of the normally available tools to do so are functional. Having served as external reviewer for numerous models over the years, I have never accepted a locked model. The assessment would be partial at best and as a reviewer I could not assert that I had determined the accuracy and validity of the model. I cannot believe that any serious reviewer of a model would agree to such a restriction.
24. The only reasons to lock a model are to protect it from inadvertently being modified in error – not a concern in this case – or because the developer fears that reasonable alternative assumptions and inputs (in a sensitivity analysis or its ilk) will reveal how weak the model and results really are.
29. In order to understand the impact of changes made on behalf of NICE to inputs and assumptions and to assess the validity of the resulting estimates, we examined the two models in detail [i.e. the original AHEAD model and the model as modified for NICE] and attempted to recreate the modified version, using our AHEAD model. In this effort, we spent several weeks involving the entire, original modelling team. We also ran AHEAD with the NICE inputs and modified it to correspond with the NICE version as much as possible. Despite having intimate knowledge of the AHEAD model, we were unable to replicate the results considered by NICE.
35. Economic modelling must be sufficiently valid to properly inform decision-makers. This requires that the model be verified to exclude errors as much as possible and that those affected by the decisions be allowed to assess the model fully – including changing inputs to other reasonable values …."
"(a) The IPR for all models rests with the institution of the author/s
(b) That while NICE may receive an executable model, it may not transmit that model to anyone else without the explicit consent of the holder of the IPR.
(c) That NICE may transmit a non-executable model to consultees subject to the usual conditions …"
Professor Sally Davies, Director General of Research and Development at the Department of Health, says in her witness statement that "economic models are commissioned on the basis that the executable version would not be disclosed". In his witness statement, Professor Andrew Clegg, a professor at Southampton University and Director of SHTAC, expresses the opinion that economic models are confidential, states that it was the University's expectation that the executable model would not be released to consultees, and confirms that "beyond the already agreed release of models to NICE in confidence, and of the non-executable model to consultees on the terms detailed above, the University did not agree to release of the model". He concludes by saying that the University considers that its agreement to release is necessary.
"Each Party shall at all times use its best endeavours to keep confidential (and to procure that its employees and agents shall keep confidential) any confidential information which it or they may acquire in relation to the business and affairs of the other Party to this Agreement and shall not use or disclose such information except with the written consent of that other Party or in accordance with the order of a court of competent jurisdiction."
The duty of confidentiality in respect of the Contractor's business and affairs is the only express obligation of confidentiality on the Secretary of State. There is no express duty of confidentiality restricting the use or disclosure of the model; and there is no reason why, in the circumstances, such an obligation might be implied so as to prevent the disclosure that Eisai seeks. It would, as Mr Pannick submitted, be very surprising if a model commissioned and paid for by the Secretary of State for the purposes of NICE's appraisal process were subject to obligations of confidentiality preventing disclosure of the fully executable version to consultees.
"32. In my opinion manufacturers in particular want to have access to, and to use, the assessment group's executable models so that they can run this process in reverse. I believe that they wish to begin with a favourable cost effectiveness figure that they believe NICE will accept, in other words, with the output, and then reverse the model and work backwards to see what combinations of inputs and assumptions will deliver that outcome. Unlike their claims for consultation and error checking, this truly is something for which they must have the executable model (and indeed is the only reason which I can see why they would need such a model, and hence my belief as to their motivations). The overall intention, of course, is to maximise profit from sales to the NHS. That is a legitimate aim for a commercial company, but it is not an aim which NICE is required to facilitate.
33. Clearly, 'mining' a model in this way may be desirable commercially but it has no intellectual validity at all. Starting from the conclusion you want to reach and working backwards is the exact opposite of the scientific approach. For the purposes of understanding the clinical and cost effectiveness of a treatment it would be a complete waste of time.
34. However, the results of such investigations would not be presented to NICE in this way. Instead NICE would have to devote a great deal of time and resource to understanding the various amendments made to its model, and to investigating the claimed basis for whatever inputs and adjustments it was that gave a supposedly cost effective outcome. Given that consultees can already advance whatever views of the evidence they wish, and that, so long as these are at least superficially credible they will be investigated by the NICE (as they were in this case), it would add nothing except cost and delay to allow the assessment group economic model to be used in this way."
"127. As the Claimant's witnesses correctly assert, the analysis of an economic model is not a trivial task. It requires time, expertise and money. In a world where resources were unlimited this would not matter, but NICE's resources are finite. It is one thing to devote time and resource to understanding the manufacturers' own approaches to economic modelling. It would be quite another to devote yet further resources to unpicking manufacturers' modifications to the assessment group modelling …. In my view, it would not be practicable for NICE to rigorously analyse all analyses that would be submitted by consultees, if NICE allowed for an executable model to be used for that purpose. The appraisal process would simply grind to a halt."
The cross-appeal on costs
Lord Justice Jacob :
Lord Justice Tuckey :