[Title] Climate Change Assessments, Review of the Processes & Procedures of the IPCC--Executive summary
Climate change is a long-term challenge that will require every nation to make decisions about how to respond. The Intergovernmental Panel on Climate Change (IPCC) was established by the World Meteorological Organization and the United Nations Environment Programme to help inform such decisions by producing comprehensive assessments of what is known about the physical climate system, its global and regional impacts, and options for adaptation and mitigation. Sitting at the interface between science and politics, the IPCC assessment process has sustained a working dialog between the world’s governments and scientists since its inception in 1988. Representatives of 194 participating governments agree on the scope of the assessment, elect the scientific leaders of the assessment, nominate authors, review the results, and approve the summaries written for policymakers. More than a thousand volunteer scientists evaluate the available scientific, technological, and socioeconomic information on climate change, and draft and review the assessment reports. The thousands of scientists and government representatives who work on behalf of the IPCC in this nontraditional partnership are the major strength of the organization.
Through its assessment reports, the IPCC has gained enormous respect and even shared the Nobel Peace Prize in 2007 for informing climate policy and raising public awareness worldwide. However, amid an increasingly intense public debate over the science, impacts, and cost of climate change, the world has changed considerably since the creation of the IPCC, with major advances in climate science, heated controversy on some climate-related issues, and an increased focus of governments on the impacts and potential responses to changing climate. A wide variety of interests have entered the climate discussion, leading to greater overall scrutiny and demands from stakeholders. The IPCC must continue to adapt to these changing conditions in order to continue serving society well in the future.
The Committee’s key recommendations for improving IPCC’s assessment process are given below.
Key recommendations
The Committee’s main recommendations relate to IPCC’s governance and management, its review process, characterizing and communicating uncertainty, communications, and transparency in the assessment process. Other detailed recommendations on specific aspects of the assessment process appear in Chapters 2-4, and a complete list of recommendations appears in Chapter 5.
Governance and management
The complexity and scale of climate change research and the associated assessment task have grown significantly over the last two decades, as have public expectations regarding the assessments. Yet the fundamental management structure of the IPCC has remained largely unchanged. The IPCC management structure comprises the Panel itself, which makes decisions about the structure, principles, procedures, and work program of the IPCC; the Bureau, which is elected by the Panel to oversee the assessment work; and a small Secretariat, which supports the work of the Panel and the Bureau. The Panel makes all of its major decisions at annual Plenary sessions. However, important decisions have to be made more often than once a year, and the Bureau has too limited a set of responsibilities and convenes too seldom to meet this need.
Many organizations in the public and private sector have addressed the need for ongoing decision making by establishing an Executive Committee to act on their behalf. Similarly, the IPCC should establish an Executive Committee elected by and reporting to the Panel. An IPCC Executive Committee would act on issues such as approving minor corrections to published reports, approving modest alterations in the scope of an ongoing assessment, ensuring effective communication, and any other task specifically delegated by the Panel. To respond quickly, the Executive Committee should be relatively small with ideally no more than 12 members. Its membership would include selected IPCC leaders as well as individuals from academia, nongovernmental organizations, and/or the private sector who have relevant experience and who are not connected with the IPCC or even climate science. Their participation would improve the credibility and independence of the Executive Committee.
Recommendation: The IPCC should establish an Executive Committee to act on its behalf between
Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and three independent members who include individuals from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.
The IPCC Secretariat supports the Panel and Bureau by organizing meetings, communicating with governments, supporting the travel of developing-country scientists, managing the IPCC budget and website, and coordinating report publication and outreach. Although the number of staff has grown from four to 10 individuals, the growth in the magnitude and intricacy of the assessment task, advances in digital technologies, and new communications needs (see ‘Communications’ below) have changed the mix of skills required of the Secretariat. An Executive Director is needed to lead the Secretariat, ensure that IPCC protocols are followed, keep in touch with the Working Group Co-chairs, and speak on behalf of the IPCC. As a peer of the Working Group Co-chairs, the individual selected as Executive Director would be capable of acting on behalf of the IPCC Chair. The Executive Director would also be a member of the Executive Committee.
Recommendation: The IPCC should elect an Executive Director to lead the Secretariat and handle
day-to-day operations of the organization. The term of this senior scientist should be limited to the time frame of one assessment.
Review process
Peer review is an important mechanism for assuring the quality of reports. IPCC’s peer review process is elaborate, involving two formal reviews and one or more informal reviews of preliminary text. The first complete draft is formally reviewed by scientific experts nominated by government representatives, observer organizations, and the IPCC Bureau. Lead Authors consider the review comments and prepare a second draft, which is reviewed by the same experts as well as government representatives. Two or more Review Editors for each chapter oversee the process, ensuring that review comments and controversial issues are handled appropriately. However, the Lead Authors have the final say on the content of their chapter.
Under a tight schedule for the revision process, authors might not always consider the review comments carefully, potentially overlooking errors in the draft report that could have been caught. Some errors will be missed in any review process, but with stronger enforcement of existing IPCC review procedures, the number of errors could be minimized. Staff support and clarification about the roles and responsibilities of Review Editors would help them carry out proper oversight.
Recommendation: The IPCC should encourage
Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report. For recent assessments, some governments made the second draft available for review by national experts and other interested parties, considerably opening the review process. Although an open review potentially improves the report by increasing the level of scrutiny and widening the range of viewpoints offered, it also substantially increases the number of review comments. Drafts of the Fourth Assessment Report drew 90,000 review comments (an average of a few thousand comments per chapter), stretching the ability of Lead Authors to respond thoughtfully and fully to each. A more targeted process for responding to reviewer comments could both ensure that the most significant review issues are addressed and reduce the burden on authors, who currently must document responses to all reviewer comments. In the targeted process envisioned, the Review Editors would prepare a written summary of the most significant review issues. While the Lead Authors would prepare written responses to these issues and all other noneditorial comments, they could focus their attention on the most significant matters.
Recommendation: The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received. Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all noneditorial comments, and no written responses to editorial comments.
Characterizing and communicating uncertainty
Uncertainty is characterized and communicated by describing how much is known about a topic (i.e., the quality and nature of the evidence available) and the probability that a particular event will occur. Each key conclusion in the Summaries for Policymakers is accompanied by a judgment about its uncertainty. For the fourth assessment, each Working Group used a different variation on IPCC’s guidance to describe uncertainty. Working Group I relied primarily on a quantitative likelihood scale (e.g., ‘extremely likely’ indicates a greater than 95 percent probability that a particular event will occur). Working Group II relied primarily on a quantitative confidence scale (e.g., ‘high confidence’ indicates an 8 out of 10 chance of being correct). Working Group III relied exclusively on a qualitative level-of-understanding scale (i.e., understanding is described in terms of the amount of evidence available and the degree of agreement among experts). The level-of-under- standing scale is a convenient way of communicating the nature, number, and quality of studies on a particular topic, as well as the level of agreement among studies. It should be used by all Working Groups, as suggested in the IPCC uncertainty guidance for the Fourth Assessment Report.
Recommendation: Each Working Group should use the qualitative level-of-understanding scale in its Summary for Policymakers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.
The Working Group II Summary for Policymakers has been criticized for various errors and for emphasizing the negative impacts of climate change. These problems derive partly from a failure to adhere to IPCC’s uncertainty guidance for the fourth assessment and partly from shortcomings in the guidance itself. Authors were urged to consider the amount of evidence and level of agreement about all conclusions and to apply subjective probabilities of confidence to conclusions when there was high agreement and much evidence. However, authors reported high confidence in some statements for which there is little evidence. Furthermore, by making vague statements that were difficult to refute, authors were able to attach ‘high confidence’ to the statements. The Working Group II Summary for Policymakers contains many such statements that are not supported sufficiently in the literature, not put into perspective, or not expressed clearly. When statements are well-defined and supported by evidence—by indicating when and under what climate conditions they would occur—the likelihood scale should be used.
Recommendation: Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of welldefined outcomes only when there is sufficient evidence. Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).
Communications
Communicating the results of IPCC assessments is challenging because of the range and complexity of climate science and response options and the increasing need to speak to audiences beyond scientists and governments. The communications challenge has taken on new urgency in the wake of recent criticism regarding IPCC’s slow and inadequate responses to reports of errors in the Fourth Assessment Report. Such criticism underscores the need for a media-relations capacity that enables the IPCC to respond rapidly and with an appropriate tone to criticisms and concerns that will inevitably arise in such a contested arena. In addition, IPCC leaders have been criticized for making public statements that were perceived as advocating specific climate policies. Straying into advocacy can only hurt IPCC’s credibility. A comprehensive communications strategy is needed to identify who should speak on IPCC’s behalf and to lay out guidelines for keeping messages within the bounds of IPCC reports and mandates. IPCC’s new communications and media relations manager is developing a communications strategy, and the Committee urges its rapid completion.
Recommendation: The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and that includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.
Transparency
Given the high stakes in climate change decision making and IPCC’s role of providing policy-relevant information, the IPCC can expect that its reports will continue to be scrutinized closely. Therefore, it is essential that the processes and procedures used to produce assessment reports be as transparent as possible. From extensive oral and written input gathered by the Committee, it is clear that several stages of the assessment process are poorly understood, even to many scientists and government representatives who participate in the process. Most important are the absence of criteria for selecting key participants in the assessment process and the lack of documentation for selecting what scientific and technical information is assessed. The Committee recommends that the IPCC establish criteria for selecting participants for the scoping meeting, where preliminary decisions about the scope and outline of the assessment reports are made; for selecting the IPCC Chair, the Working Group Co-chairs, and other members of the Bureau; and for selecting the authors of the assessment reports. The Committee also recommends that Lead Authors document that they have considered the full range of thoughtful views, even if these views do not appear in the assessment report.
If adopted in their entirety, the measures recommended in this report would fundamentally reform IPCC’s management structure while enhancing its ability to conduct an authoritative assessment. However, no matter how well-constructed IPCC’s assessment practices may be, the quality of the result depends on the quality of the leaders at all levels who guide the assessment process. It is only by engaging the energy and expertise of a large cadre of distinguished scholars as well as the thoughtful participation of government representatives that high standards are maintained and that truly authoritative assessments continue to be produced. Moreover, the IPCC should think more creatively about maintaining flexibility in the character and structure of the assessment, including the number and scope of Working Groups and the timing of reports. For example, releasing the assessment of regional impacts substantially after the assessment of sectoral impacts would reduce the burden on the small community that carries out both assessments. It may also be desirable to release the Working Group I report a year or two ahead of the other Working Group reports. Although such issues are routinely raised and settled in the scoping process, the traditional approach may not be the best model for future assessments.