Martin Lally
Some thoughts on the Royal Commission Covid-19 Report released last Tuesday, focusing upon the 2021 lockdowns and the vaccine:
1. The goal of the Report was to assess the decisions that were made, which involved four primary issues:
(a) Was the government’s decision-making process sound?
(b) Were the lockdown decisions in late 2021 correct?
(c) Was the decision to mandate the vaccine in late 2021 correct?
(d) Is the vaccine safe?
2. In respect of the government’s decision-making process, the Report is replete with significant criticisms. For example, para 1009 states that “..while decision-makers in the covid-19 pandemic made good use of frequent, detailed health advice on how public health measures would affect the transmission of the virus and rates of hospitalisations and death, the quality, coherence and timeliness of formal advice on economic, social and educational impacts did not in general match that of advice on health impacts. Para 475 is even blunter in respect of the education mandates: “There was no assessment in the cabinet paper of the potential scale of opposition and implications for education providers, learners and their families, nor of the income and employment effects for people who declined vaccination. The employment and wage scarring effects of involuntary unemployment in New Zealand are well known and could have been estimated.” Para 476 goes on to say that “As it turned out the employment and wage scarring impacts of mandates in New Zealand were significant.” Para 504 states that “Cabinet papers and briefings to ministers do not explicitly refer to whether the restrictions on rights imposed by the vaccine requirements were proportionate to the public health benefit. Nor do they indicate whether any less restrictive options that could achieve that benefit had been considered.” These are significant criticisms.
3. In respect of the lockdowns in late 2021, the Report noted the government’s view that the strategy with the best public health outcomes would also have the best economic and social outcomes (para 877). The Report concurs with this view conditional on the virus being eliminated quickly but added that it is was not clearly true otherwise and that by the end of October 2021 (with the Auckland lockdowns having failed to eliminate the virus by then) the strategy ought to have been abandoned (para 922-925). In particular, para 925 states that “a more structured approach to strategy review could have identified the change in circumstances and allowed for consideration of a different strategy sooner.” It might seem uncontroversial that, if one can eliminate the virus quickly, doing so is optimal because it minimises all of the adverse effects. However, if a lockdown is initiated, one cannot know at the time of doing so whether it will succeed. Furthermore, if it failed to eliminate the virus in (say) one month, there will be pressure to continue arising from the sunk cost fallacy: ‘we can’t give up now or all of our past efforts will be wasted’. So, in deciding whether to lockdown, one would have to form some view on the probability of success and in the event of failure how long the lockdowns would continue before the government (likely suffering from the sunk cost fallacy) would accept that this had happened. The Report has nothing to say on these two parameters, and does not suggest that the government estimated them either. Thus, while the Report offers no view on whether the initial decision was sensible, it faults the government for not reversing course in late October 2021.
4. In respect of the decision to mandate the vaccine in late 2021, the Report notes the significant adverse effects upon those who refused the vaccine but does not reach a conclusion as to whether the decision was justified. Its analysis of this issue finishes (para 522) with “Vaccination requirements are a valid intervention that should be kept in the toolbox for future pandemic responses. However the scale of their intrusion on fundamental rights and freedoms and their potential for severely negative impacts on those who choose not to be vaccinated means they should be used with great care.” This is not an endorsement of their use in 2021 and seems like a veiled criticism of it.
5. The Report notes at many points that decisions in this area involved balancing public health impacts against economic, social and educational impacts, and the Terms of Reference required the Commissioners to (inter alia) assess whether the decisions struck a reasonable balance (para 183). Since these different impacts are measured in different ways, it is standard practice within the public sector, health economics and medicine to convert all of these effects into Quality Adjusted Life Years (QALYs), and then into dollars (by using a monetary value for a QALY) if there are also financial impacts, and this is a type of cost-benefit analysis. For example, The Treasury does so: see
and Chapter 4 contains a detailed example of a cost-benefit analysis relating to a vaccine, with the benefits expressed in terms of QALYs and then converted to dollars. The large team assisting the Commissioners included people who were aware of this standard process, and there were also submissions on this matter. Despite this, there is not a single use of the term QALY or even reference to the concept in the entire 700 pages of the Royal Commission’s Report. Even the phrase “cost-benefit analysis” appears only three times in the report, and then only in passing rather than as a precursor to actually doing it. So, in attempting to assess the government’s decisions, the Report did not use the essential tool for determining the overall effect of the decisions and this must have been deliberate rather than lack of awareness of the standard methodology. The Report records the fact that there were public health, economic, social and educational effects, but it does not formally determine the net effect using QALYs or any other method. The same appears to be true of all of the government’s pandemic decisions. It is rather like someone being appointing to assess the financial viability of a business and noting that the business earned revenues from activities A and B, and incurred costs of types C and D, but never expressing these in the common unit of dollars so that the costs could be deducted from the revenues to determine the net effect (profit or loss). Merely listing the various types of revenues and costs, and then using one’s judgement to assess whether the former outweighed the latter, would not substitute for determining the business’s profit or loss. The same is true of failing to convert the multifarious consequences of a public health intervention into QALYs.
6. To illustrate how this QALY methodology could be used in the present context, consider the issue of the vaccine mandates. The earlier 2024 Royal Commission Report noted that the first-dose vaccination rate (for the 15+ group) was about 80% when the first mandates took effect at the end of October 2021 and plateaued at about 90%, so the mandates could only have raised the vaccination rate by 10% at most: see Figure 3 on page 343 of https://www.covid19lessons.royalcommission.nz/reports-lessons-learned/main-report
In addition Table 1 of the following paper estimated that vaccinating the last 10% saved 600 lives (3,788 v 3,163):
The Commissioners were well aware of this paper because one of its authors (Prof Michael Plank) summarised it in an oral presentation to the Commissioners. In addition, covid victims had an average residual life expectancy of about five years: see
So, up to 600*5 = 3,000 QALYs were saved by the mandates. This is the benefit of the mandates. The cost is the impact of the penalties for non-vaccination on those who refused. Across the 4 million New Zealanders of age 15+, the 10% who refused vaccination (those above the plateau rate of 90%) then constituted about 400,000 people. As noted in para 513 of the Report, “the vaccination requirements had significant negative social and economic impacts for people who declined vaccination”. These included significant loss of social relationships (possibly permanent), possible dismissal by their employer (and about 40% of workers were subject to employment mandates: para 489 of the Report), difficulty in securing alternative employment on account of the reason for dismissal, been denied various commercial services for several months, and been stigmatised as an “antivaxxer”.
It is necessary to express these adverse impacts on the 400,000 vaccine refusers in QALY terms, to compare them with the mandate benefits. Using the Wellbeing (WELLBY) framework, which is used by Treasury (see the Treasury document in para 5 above), Frijters estimated normal life at 6 WELLBYs, the loss of employment as a reduction of 0.7, and forced social distancing and lockdowns of the UK type as a loss of 0.5. See page 53 of
Since a year of life in good health is one QALY, the loss of employment for one year would be equivalent to the loss of 0.7/6 = 0.12 QALYs and forced social distancing for one year would be equivalent to the loss of 0.5/6 = 0.08 QALYs. Given the adverse impacts on the vaccine refusers noted above, the aggregate impact per person would likely have been in excess of that of a person who experienced loss of employment (0.12 QALYs per year) or was subject to lockdown (0.08 QALYs per year) because these latter two estimates do not reflect allowances for the denial of various commercial services, social stigma, and suffering the loss of both social relationships and employment. So, a conservative estimate for the adverse effect on vaccine refusers would be 0.08 QALYs per year.
Conservatively assuming the loss persists only for the six month average duration of the mandates implies a loss of 0.04 QALYs per vaccine refuser. Multiplication of this by the 400,000 refusers yields 16,000 QALYs lost, which far exceeds the QALYs gained of less than 3,000. So, the mandating decision does not seem justified.
7. Despite failing to endorse either the late 2021 lockdowns or the vaccine mandates, the Report states “On the whole the decisions taken and methods used during the Covid-19 response were considered and appropriate.” (page 4). Since the claim concerning “appropriate” is so much at variance with the main body of the Report pointedly not endorsing either the lockdown or mandate decisions, it may have been intended to soften the blow to the government from declining to endorse those decisions, and to soften the blow to the public’s trust in their competence. Alternatively, different messages may have been designed for different audiences. Those who read only the summary (most politicians and journalists perhaps) would see an endorsement of the government’s decisions whereas those who read the entire report would see a veiled but strong criticism of those decisions.
8. The Report also states in para 18 that “ministers and officials were facing a series of complex, high stakes decisions in a rapidly changing environment and were doing the best they could at the time.” However, the best that they could do would be to use the cost-benefit analysis that was standard practice in the public sector, which is referred to in para 5 above, and this was not done. So, ministers and officials were not doing the best that they could. The fact that the decisions were complex and high stakes magnified the importance of using this standard methodology, and therefore magnified the errors resulting from not doing so. Again this statement in the early pages of the report conflicts with the main body of the Report pointedly not endorsing either the lockdown or mandate decisions. Again, this is softening the blow or providing different messages for different audiences.
9. The Report’s recommendations on page 29 include “develop a framework for structuring advice to decision makers in a future pandemic that enables them to effectively weigh impacts of public health measures across desired outcomes and so to identify the important trade offs”. Such a framework already existed and it has done so for some time. It is called the Treasury CBAx Tool User Guidance, and is referred to in para 5 above. Again, the large team assisting the Commissioners included people who were aware of this standard process. So, while most readers would find the recommendation indecipherable, those familiar with cost-benefit analysis would understand that this was a rebuke to those involved in the decisions for failing to use the CBAx methodology.
10. In respect of whether the vaccine was safe, the Report is much clearer. It devotes an entire appendix to examining submissions arguing that the vaccine was unsafe. In doing so it referenced a large body of scientific literature and concluded that “the concerns raised are not grounded in reliable evidence or aligned with scientific consensus.” (para 1383) These issues are complex, as reflected in the vocabulary used in this Appendix: “oncogenes”, “retrotransposons”, “reverse transcriptase”, and the “plasmid template”. Since the Commissioners comprise two people with legal training and one with economics training, it is unlikely that this appendix represents their views because it is unlikely that they had a view on all or even most of the matters there. If most of the appendix is beyond the knowledge of the Commissioners, and most of it is beyond mine, it must then have been authored by people with expertise in this area. These authors should have been revealed so that readers of the Report could assess their credibility, just as it was important for the names of the Commissioners to be revealed. Ironically, despite this lack of disclosure, the Report recommends on page 34 that the government be “open and clear about where they are obtaining their evidence from.”
11. Whoever these unidentified authors of the appendix are, the task of reviewing a complex set of scientific papers would require a high level of expertise and anyone with that expertise would most likely have already formed a view on the safety of the vaccine by 2025, and therefore on the merits of any submissions arguing that it was unsafe. Thus, in choosing relevant experts to assess the submissions, the choice open to the Commissioners was limited to experts who already believed the vaccine to be safe to an acceptable level and those who thought otherwise. Had they assigned the task to experts of the latter type, the result of the review would then most likely have been to state that the vaccine was unsafe. Quite apart from the resulting torrent of vilification that the Commissioners would have faced, such an outcome would have triggered massive anxiety amongst those who had been vaccinated (which is most of us) and a major erosion of trust in the system. So, by default, the experts to whom this task was entrusted would have to be those who already believed that the vaccine was safe to an acceptable level. The outcome would then have been essentially preordained. This does not mean the outcome was wrong, and I hold no view on that question because I don’t have a sufficient knowledge of the science to assess these matters. Nor is there obviously a better way of dealing with this matter, but the value of that appendix is significantly reduced by the fact that all of those fit to assess the question would most likely have already formed a view on it by 2025, and the outcome of the review was then predetermined by the choice of reviewers. Similarly, if a Committee of experts was tasked with assessing whether globalisation was beneficial to most people, or whether New Zealand should have a capital gains tax, the outcome would be determined by the choice of experts.
12. This problem of choosing people to assess a matter after they have already formed and expressed their view is aggravated by the intense emotions that the pandemic has unleashed, even among the scientific community. In respect of the lockdowns, many scientists were opposed to them, and some of them therefore supported the Great Barrington Declaration. Others supported the lockdowns and some of them therefore supported the John Snow Memorandum. The Wikipedia page on the Great Barrington Declaration quotes the views of some scientists who opposed the Declaration:
These contrary views include
“fringe epidemiology” (F. Collins)
“culling the herd of the sick and disabled” (G. Gonsalves)
“what everyone really thinks is this is fucking stupid” (R. Beale)
“a dangerous mix of pixie dust and pseudoscience” (M. Osterholm)
“a form of astroturfing similar to creationism” (D. Gorski)
“an argument for eugenics” (D. Gorski)
Scientists who write in this way have clearly become too emotionally entangled in the subject to be relied upon to assess evidence objectively. The undeclared authors of the appendix have not used phrases like this in the appendix but, without disclosure of their names, it cannot be assessed whether they have elsewhere expressed such emotive views on the matter as to disqualify them from the task.
13. The effort devoted to rebutting concerns about the vaccine contrasts strikingly with the complete absence of any cost-benefit analysis of the mandating and lockdown decisions, especially since the latter decisions were made by the New Zealand government while responsibility for the safety of the vaccine primarily lies with Pfizer and the FDA. While a total of 32 pages were devoted to assessing concerns about the vaccine the Report failed to devote even one paragraph to cost-benefit analysis of the 2021 lockdown and mandating decisions, such as that in paragraph 6 above (which strongly suggests that the mandates were unwarranted). As noted in para 5 above, the large team assisting the Commissioners included people who were well aware of cost-benefit analysis in the public sector and there were submissions on this matter. So, the Report’s failure to apply standard public-sector cost-benefit analysis to these decisions must have been deliberate rather than lack of awareness of this methodology. This might be because the Commissioners felt that any such cost-benefit analysis of the mandates and the 2021 lockdowns might definitively conclude that they were not warranted and, if so, presentation of that conclusion in the Report might trigger a wave of legal actions for compensation and a resurgence of the grave social fractures that resulted from these government decisions. To avoid this outcome, the Commissioners instead chose the veiled criticisms noted in paras 3 and 4 above.
14. In summary the Report:
(i) concluded that the government’s decision-making process was unsound in many respects, most particularly in focusing upon the public health consequences of its decisions at the expense of the social, economic and educational consequences.
(ii) concluded that the Auckland lockdowns were maintained well past the point at which it was clear that they had failed, but did not offer a view on whether the imposition of these lockdowns was warranted.
(iii) did not endorse the vaccine mandates in 2021 and appeared to criticise their use in 2021.
(iv) did not itself apply standard cost-benefit analysis to the government’s decisions, which would have required converting deaths, social impacts, and educational impacts into QALYs in accordance with standard practice in the public sector.
(v) concluded that concerns about the safety of the vaccine were not supported by reliable evidence, but the complexity of these issues is such that this would be the view of the experts appointed to do so rather than the Commissioners, and the views of these experts would very likely have been known before they even commenced their review.
So, leaving aside the issue of the safety of the vaccine (for which credit or blame will lie primarily with Pfizer and the FDA), and notwithstanding the Report commencing with the statement that the government’s decisions and methods were “appropriate”, the detail within the Report is instead quite critical of both the government’s methods and decisions.