Strand, London, WC2A 2LL
B e f o r e :
| LION APPAREL SYSTEMS LIMITED
|- and -
Mr Michael Bowsher QC & Mr Rob Williams (instructed by Pinsent Masons) for the Defendant
Hearing dates: 18th, 19th, 20th, 21st, 24th & 25th September 2007
Crown Copyright ©
Mr Justice Morgan :
THE PROCUREMENT PROCESS
THE RELEVANT LEGAL PRINCIPLES
THE ORDER IN WHICH I WILL CONSIDER THE HEADS OF CHALLENGE
- Bristol's criminal convictions;
- Scoring methodology;
- The technical files;
- The non-structural tunic;
- The call centre;
- Tier1/ Tier 2 availability;
- Brigade stock;
- Fitting of garments;
- Feedback on garments;
- Feedback at the BDD relating to price;
- Breach of confidentiality;
- Legal mark up;
- General favouritism.
BRISTOL'S CRIMINAL CONVICTIONS
"Providing false information will invalidate this application and may disqualify an organisation from consideration."
"I understand and accept that false information could result in rejection of the Company's application to tender."
"we did not get a full and frank response from Bristol straightaway."
"I have to say that I am surprised that that is now how Lion puts its case. Although I have had very little time to prepare substantial evidence to rebut the proposition, I must say that in the thirty years I have worked in Government, much of it dealing with procurement, I have never before had such a proposition suggested to me. It would lead to the conclusion that all procuring authorities would be bound to select and limit themselves to one scoring methodology for all different areas within a particular tender, whether or not that methodology was in their view the most appropriate way of establishing which tender submitted the most economically advantageous tender.
In my experience, it is quite usual to adopt different scoring methodologies in different assessment areas in the way that was adopted by the ICP. The reason for this is, I would suggest, quite obvious and quite rational. It is sensible and usual to adopt a linear scoring progression in relation to price, as a linear scale will capture and appropriately reflect the differences between bidders' prices. However, in relation to other parts of an assessment, it may be necessary to adopt some other approach to capture relevant differences and to differentiate effectively between bidders. That is particularly so where the results of that part of the assessment are likely to fall in a narrow range when ranked on a linear scale, and where a linear scale would not properly differentiate between bidders on the characteristics being assessed. Examples from my personal experience include the following: procurement of occupational health services, scientific services, consultancies and legal services. In each case, price has been amenable to assessment on a linear scale with the different technical characteristics of different health providers (which may have different solutions to the same procurement issue, for example, by providing different facilities and a different number of staff), consultancies, scientific service providers and law firms being scored on a different basis to provide a mechanism that distinguishes between different bids. In each of these cases (as with the comfort and dexterity of garments – see below) there is a subjective element which needs to be assessed and differentiated.
Applying that to the ICP, it was as I have said entirely logical and appropriate to adopt a linear scoring system on price. However, in relation to the garment trials, "raw" test data had to be translated into some sort of scoring system in order to facilitate the evaluation of different bids. There were obviously a range of recognised scoring systems available to the ICP. The reason for adopting a modular scoring system was twofold. First, the differences between measured performance in relation to garments sometimes appears very small, and results can therefore become clustered. However, those apparently small differentials can translate into much more significant and important differences in performance, in many instances with important implications for fire fighter comfort, dexterity and health and safety. In order to ensure that these effects are recognised it is important to adopt a scoring system which differentiates effectively between results. A modular system does exactly that.
A second and related point is that some of the matters being assessed, such as comfort and dexterity, were necessarily assessed on a qualitative basis, and a modular system is obviously suitable for the assessment of such matters, not being capable of precise measurement.
I should also point out that the methodology used was devised by the GTWG which consisted of a number of technical experts within the fire service and was endorsed by the full ICP Board. In other words, full consideration was given to what approach should be taken by a significant number of appropriately qualified technical staff.
It is right to say that this approach translates a certain quantified value into a ranking or position, and I understand this to be one of Lion's criticisms. As I have explained, this is a common and orthodox scoring technique adopted for example where it is thought that a raw quantified differential will not give adequate visibility to important differences between bids. I have made such inquiries as I have been able to make in the time available with a view to demonstrating that this is a widely held view, and far from "unique" or "special" as Mr Hirst has suggested to the Court. To that end, I have asked Peter Marshall, professional services manager at Commerce Decisions Ltd, a consultancy providing expert assistance in relation to evaluation and management of procurement projects and subsequent contract management ( … ) to comment on his experience of bid evaluations that incorporate different methods of scoring for different areas of the evaluation. He has written an email to me today to explain and confirm that multi-methodology scoring approaches are common in procurement in this country."
"Some examples of the varieties of scoring methodology which we have experienced are listed below.
In most evaluations of "complex" goods and services it is necessary and desirable to investigate the value (to the acquirer) of the technical solution being proposed (e.g. its "capability", "utility" or "compliance"), the price/cost of the solution and terms associated with its acquisition.
Whilst not universal, it is common practice for our clients to combine the price/cost and technical parts of a supplier evaluation in order to derive an overall score for each bid. It is common practice to call this overall measure the "value for money" of the proposal. This is often done as well as measuring the "compliance" of each proposal against a set of pre-defined minimum technical or commercial requirements.
It is common practice, and I believe entirely appropriate, to combine the technical and price/cost aspects of an evaluation to derive an overall score. This is often done even where the technical and price/costs aspects have been evaluated using different scoring methodologies.
It is also common practice to score the technical part of an evaluation against a numerical or adjective based scale with a number of pre-defined or calibrated points designed to measure the "capability", "utility" or "compliance" of the solution. An example would be a 0-10 scale with definitions against some, or all, of the points. Another example would be the four point scale: "Excellent"; "Adequate"; "Barely Adequate" or "Not adequate". This type of evaluation assesses bids by allocating to them a score at one of the defined points on the scale (10 or 9, or Excellent or Adequate), but not allowing part scores (such as 9.5). A bidder can only achieve a score at one of the pre-defined points on the scale.
It is also common to convert an evaluation of actual "real world" performance data to a numerical or adjective-based scale. This occurs and is in my view legitimate even in cases where the conversion to a numerical or adjective-based scale will result in scores for various bidders with a higher comparative difference that (sic) the underlying real-world performance data. This might be desirable to achieve a particular effect, such as to achieve differentiation between bids on a particular aspect where there are only likely to be small differences between the real world performance data of the various bidders.
When measuring price or cost it is almost universal practice to measure it in absolute currency terms (e.g. "unit acquisition cost" stated in £, or "whole life cost" stated in £, or "risk-adjusted price" stated in £.)
When combining the technical and price/cost parts of an evaluation it is therefore nearly always necessary to find a way of converting the price/cost into a score so that this score can be combined with the technical score in accordance with their relative weights. There are many methods used to conduct this conversion; a common one used is to award the cheapest bid a 100% score for the price/cost criterion and every other bid a proportional score (e.g. a bid that is twice as expensive would be awarded a 50% score for the criterion)."
"In Mr Robinson's first statement, he makes certain criticisms which relate to the way in which ensembles were scored. In that regard, I would like to draw attention to certain passages in the new British Standard BS 8469 (2007) – "Personal protective equipment for firefighters – assessment of ergonomic performance and compatibility-requirements and tests methods". As Dr Claire Millard noted in her evidence … parts of the ICP garment tests have been incorporated into that standard. The British Standards Institute ("BSI") is an independent national body which represents the UK view on standards in Europe and internationally … . It provides an independent view of the best methods for (in this case) testing of PPE.
The significance of this standard is that the BSI has considered the methodology adopted by the ICP in a detailed, independent review, and concluded that that methodology was sufficiently robust that it would endorse it. It says that:
"The general approach and the testing methodology included here has been developed under the DTI Consultancy Drafting Scheme, and incorporate practical experience gained during a major contemporary assessment of firefighters' PPE ensembles [i.e. the ICP]"
The BSI goes on to make a number of observations which are pertinent to various allegations made by Lion in this proceeding. In BSI's view, testing of ensembles is preferable to testing individual elements. It concludes:
"This standard has therefore been prepared to enable firefighters' PPE ensembles to be evaluated and objectively assessed for ergonomic performance as a complete ensemble, rather than in their component parts…."
"It specifies the testing of individual items of the PPE as an ensemble so that the compatibility of the individual items can be evaluated and any adverse interactions between the individual items can be identified."
MY CONCLUSION ON SCORING METHODOLOGY
THE TECHNICAL FILES
THE NON-STRUCTURAL TUNIC
"Clarification Question 55 told us that the PPE issue per Fire Fighter for non-structural PPE was TBA and implied that we should include non-structural tunic only. Is it FiReBuy's intention that a non-structural tunic only should be included in the model for the categories AB/AC/AD? Additionally, for the National price per Firefighter, it means that FRAs specifying no Wildland or Technical Rescue/USAR kit are paying for those that have".
"The point raised about FRAs not requiring these additional PPE categories subsidising those that do is well made and that was certainly not FiReBuy's intention. Bidders should exclude all non structural PPE clothing items from the FMS financial model but where they have previously included them, they may show the marginal cost to FRAs as an option separately from the Unitary Charge".
"Lion's original pricing in respect of Fully Managed Service (FMS) options for both the Mandatory Standard Bid (MSB) and Variant Bid options have been changed to show the price relationship on a price per slot per day basis in respect of "Structural" PPE Category AA and the other "Non-Structural" PPE Categories AB-AC-AD.
As can be noted this offers a considerable price reduction for the "Structural" PPE category AA option for those FRAs not requiring wildland and/or rescue options. This does not of course reduce the overall pricing metric as it would be questionable to now do so at this stage of the Tender deliberations.
Re-presenting the Lion offer and showing the pricing differential between the respective PPE Categories in this way, the preferred options can be more readily assessed by the Contracting Authority and of course by FRAs in due course.
In respect of MoDFS the overall Bid price remains unchanged albeit the service delivery model and scale of PPE provision has been reviewed to meet the recently confirmed revised requirements of the Service (Refer to: SD ROR 13.31 revised)."
" … Firebuy has also considered the maximum potential implications of the issue for Lion's score. I note that in paragraph 40 of his statement, Mr Robinson claims that the issue would have affected its score by 3.2% overall. There are no workings set out in relation to this figure, and so I am unable to comment on how it is arrived at, but according to our calculations, this greatly overstates the significance of the issue.
TTHB5 at pages 4 to 5 sets out a calculation prepared by Mr Critchley, showing the maximum possible implications of this point. On the basis of that calculation, one can see that if the point is correct, and no change is made to any other bidder's score, the point would result in a 1% reduction in margin between Lion and Bristol (through a reduction to Bristol's price score. A similar process would have to be adopted in relation to [the underbidder's] score, which would result in a similar reduction).
On that basis, one can see that this issue could not have made any difference to the outcome of the competition, as the margin between Lion and Bristol was 5.6%. Indeed, Lion would still have remained in third place behind Bristol and [the underbidder]. To put the point another way, although Lion has now had a very substantial opportunity to analyse critically the scoring of its bid relative to that of Bristol, and to seek to upset Firebuy's decision, nothing in Lion's complaints causes me to doubt the robustness of our overall conclusion, which was that Bristol was the successful bidder by a clear margin."
THE CALL CENTRE
TIER 1/TIER2 AVAILABILITY
"Bidders should not necessarily price against the principles of Tier-1 and Tier-2 availability…".
"Bidders will recall that they were asked to comply with the spirit of the availability mechanism proposed within the PKF paper rather than the specific requirements of Tier-1 and Tier-1 availability."
"It would be prudent to assume that there will be minimal, if any, support from FRAs in achieving the availability criteria as proposed by Bidders. Where Bidders feel that support would be required from individual FRAs and/or the Contracting Authority, then these assumptions should be stated in full to enable the Contracting Authority to take the resulting impact into account during the evaluation phase."
FITTING OF GARMENTS
FEEDBACK ON GARMENTS
FEEDBACK AT THE BDD RELATING TO PRICE
"…is it fair that bidders are being given a seond bite of the cherry, but in particular, that each bidder is being told which areas it must improve on?"
BREACH OF CONFIDENTIALITY
STANDING BACK ON POSSIBLE CHANGES IN SCORING
LEGAL MARK UP
NOTICE UNDER REGULATION 32 OF THE 1993 REGULATIONS
PROMPTNESS OF ISSUE OF PROCEEDINGS
ADEQUACY OF DAMAGES FOR LION
ADEQUACY OF DAMAGES FOR FIREBUY AND OTHERS
BALANCE OF CONVENIENCE