Climatic Research Unit documents


Climatic Research Unit documents including thousands of e-mails and other computer files were stolen from a server at the Climatic Research Unit of the University of East Anglia in a hacking incident in November 2009. The documents were redistributed first through several blogs of global warming deniers, who alleged that the documents indicated misconduct by leading climate scientists. A series of investigations rejected these allegations, while concluding that CRU scientists should have been more open with distributing data and methods on request. Precisely six committees investigated the allegations and published reports, finding no evidence of fraud or scientific misconduct. The scientific consensus that global warming is occurring as a result of human activity remained unchanged by the end of the investigations.
The incident occurred shortly before the opening December 2009 Copenhagen global climate summit. It has prompted general discussion about increasing the openness of scientific data. Scientists, scientific organisations, and government officials have stated that the incident does not affect the overall scientific case for climate change. Andrew Revkin reported in The New York Times that "The evidence pointing to a growing human contribution to global warming is so widely accepted that the hacked material is unlikely to erode the overall argument."

Content of the documents

The material comprised more than 1,000 e-mails, 2,000 documents, as well as commented source code, pertaining to climate change research covering a period from 1996 until 2009. Some of the e-mails which have been widely publicised included discussions of how to combat the arguments of climate change sceptics, unflattering comments about sceptics, queries from journalists, and drafts of scientific papers. There have been assertions that these discussions indicated efforts to shut out dissenters and their points of view, and included discussions about destroying files in order to prevent them from being revealed under the UK Freedom of Information Act 2000.
A review by the Associated Press of all the e-mails found that they did not support claims of faking of science, but did show disdain for critics. Scientists had discussed avoiding sharing information with critics, but the documents showed no evidence that any data was destroyed. Researchers also discussed in e-mails how information they had released on request was used by critics to make personal attacks on researchers. In an interview with The Guardian, Phil Jones said "Some of the emails probably had poorly chosen words and were sent in the heat of the moment, when I was frustrated. I do regret sending some of them. We've not deleted any emails or data here at CRU." He confirmed that the e-mails that had sparked the most controversy appeared to be genuine.

E-mails

Most of the e-mails concerned technical and mundane aspects of climate research, such as data analysis and details of scientific conferences. The controversy has focused on a small number of e-mails, particularly those sent to or from climatologists Phil Jones, the head of the CRU, and Michael E. Mann of Pennsylvania State University, one of the originators of the graph of temperature trends dubbed the "hockey stick graph".
Climate change deniers gained wide publicity for allegations that the hacked e-mails showed climate scientists colluded in manipulating data, withheld scientific information, and tried to prevent dissenting scientific papers from being published in peer reviewed journals. Academics and climate change researchers said that nothing in the emails proved wrongdoing, and dismissed the allegations. Independent reports said that the e-mails did not affect evidence that man made global warming is a real threat, and e-mails were being misrepresented to support unfounded claims of scientific misconduct, but there were disturbing suggestions that scientists had avoided sharing scientific data with sceptical critics.
The Information Commissioner's Office stated that "the prima facie evidence from the published e-mails indicate an attempt to defeat disclosure by deleting information. It is hard to imagine more cogent prima facie evidence. ... The fact that the elements of a section 77 offence may have been found here, but cannot be acted on because of the elapsed time, is a very serious matter." The Science and Technology Select Committee criticised the ICO for making "a statement to the press that went beyond that which it could substantiate", but also blamed the university for mishandling Freedom of Information requests and called for a full investigation to resolve the questions raised.
The Associated Press conducted a review of the e-mails and concluded that they showed scientists fending off critics, but did not support claims that global warming science had been faked. They stated that "One of the most disturbing elements suggests an effort to avoid sharing scientific data with critics skeptical of global warming", and mentioned ethical problems with this action due to the fact that "free access to data is important so others can repeat experiments as part of the scientific method". They cited a science policy expert as stating that it was "normal science politics, but on the extreme end, though still within bounds". The AP sent the emails to three climate scientists they selected as moderates, who did not change their view that man-made global warming is a real threat. The three scientists are on the record elsewhere supporting an outside, independent review of the allegations of misconduct at both the CRU and Pennsylvania State University.
Summarising its own analysis, FactCheck stated that claims by climate sceptics that the emails demonstrated scientific misconduct amounting to fabrication of global warming were unfounded, and emails were being misrepresented to support these claims. While the emails showed a few scientists being rude or dismissive, this did not negate evidence that human activities were largely responsible for global warming, or the conclusions of the 2007 Intergovernmental Panel on Climate Change report which used the CRU as just one of many sources of data.
An editorial in the scientific journal Nature stated that the e-mails had not shown anything that undermined the scientific case on human caused global warming, or raised any substantive reasons for concern about the researchers' own papers. Scientific openness requires public availability of data used to reach conclusions, but researchers had been hampered in this by contractual restrictions on some data, and in certain countries national meteorological services were too slow to provide data sets on request. The e-mail theft highlighted "the harassment that denialists inflict on some climate-change researchers, often in the form of endless, time-consuming demands for information under the US and UK Freedom of Information Acts. Governments and institutions need to provide tangible assistance for researchers facing such a burden." While scientists are human and "unrelenting opposition to their work can goad them to the limits of tolerance", they "should strive to act and communicate professionally, and make their data and methods available to others, lest they provide their worst critics with ammunition."
The IPCC's chairman, Rajendra Pachauri, described the CRU's scientists "as highly reputed professionals, whose contributions over the years to scientific knowledge are unquestionable" and described their datasets as "totally consistent with those from other institutions, on the basis of which far-reaching and meaningful conclusions were reached in the ."
On 24 November the University of East Anglia issued a statement on the contents of the e-mails: "There is nothing in the stolen material which indicates that peer-reviewed publications by CRU, and others, on the nature of global warming and related climate change are not of the highest quality of scientific investigation and interpretation."
In its report to the Science and Technology Select Committee of the Parliament of the United Kingdom, the Institute of Physics stated that the emails reveal evidence of "determined and coordinated refusals" to comply with scientific traditions through "manipulation of the publication and peer-review system" and "intolerance to challenge". This report was used by climate sceptics to bolster claims that the problem of global warming is exaggerated. This forced the Institute of Physics to confirm that its position was that "the basic science is well enough understood to be sure that our climate is changing, and that we need to take action now to mitigate that change." Many experts considered that the correction was still inadequate, with climatologist Andy Russell describing the allegation of data suppression as "incorrect and irresponsible". The institute said that the statement had been prepared by their energy subcommittee, but would not reveal who had produced it. It did say that the subcommittee included an IOP official named Peter Gill, whose company provides services to the energy industry and who has written that for many people, the subject of anthropogenic global warming "has become a religion, so facts and analysis have become largely irrelevant". The institute said that Gill was not the main source of information and that other members of the sub-committee were also critical of CRU. Evan Harris, a Liberal Democrat member of the Science and Technology Select Committee, said "Members of the Institute of Physics ... may be concerned that the IOP is not as transparent as those it wishes to criticise." However the institute told the Guardian that the submission was "approved by three members of its science board" and supplied comments from an anonymous board member stating "The institute should feel relaxed about the process by which it generated what is, anyway, a statement of the obvious... the points makes are ones which we continue to support, that science should be practiced openly and in an unbiased way."

Climate reconstruction graph

The most quoted phrase took words from an e-mail of 16 November 1999 written by Phil Jones which referred to a graph he was preparing as a diagram for the cover of the World Meteorological Organization statement on the status of global climate in 1999. Jones wrote:
The "trick" was a technique to combine data series, and "the decline" was a well known issue with Keith Briffa's reconstruction using certain tree ring proxies which appeared to decline after 1950, when measured temperatures were rising. The email was widely misquoted as a "trick" to "hide the decline" as though it referred to a decline in measured global temperatures, but this was obviously untrue as when the email was written temperatures were far from declining: 1998 had been the warmest year recorded. On 9 December 2009 Sarah Palin said the truncated phrase showed a "highly politicised scientific circle" had "manipulated data to 'hide the decline' in global temperatures", and at the 2009 United Nations Climate Change Conference in Copenhagen, senator Senator Jim Inhofe quoted Jones, and said "Of course he means hide the decline in temperatures".
The graph showed three series of paleoclimate reconstructions, based on records of tree rings, corals, ice cores, lake sediments, etc., along with historical and instrumental records. "Mike's Nature trick" referred to Michael E. Mann's paper on temperature trends published by Michael Mann in Nature in 1998, which combined various proxy records and related them to actual temperature records: it included a figure later dubbed the ""hockey stick" graph, which clearly distinguished between the proxy and instrumental data. Mann described the "trick" as simply a concise way of showing the two kinds of data together while still clearly indicating which was which. He said that there was nothing "hidden or inappropriate" about it, and that his method of combining proxy data had been corroborated by numerous statistical tests and matched thermometer readings taken over the past 150 years. A press release by the University of East Anglia said that the "trick" was using instrumental data to meet a requirement of showing temperatures more recent than those covered by the proxy based temperature reconstructions, and that the use of the word "trick" was not intended to imply any deception. An editorial in Nature said that 'trick' was slang for a clever technique.
The phrase "hide the decline" referred specifically to the divergence problem in which some post 1960 tree ring proxy data indicates a decline while measured temperatures rise. The reconstruction by Briffa et al. was based solely on tree ring data, which shows a strong correlation with temperature from the 19th century to the mid 20th century. They had published a statement on the divergence problem in 1998, and had recommended that the post 1960 part of their reconstruction should not be used. Jones stated that the email was "written in haste" and that, far from seeking to hide the decline, CRU had published a number of articles on the problem. The implications of the decline are discussed in Chapter 2 of the IPCC Third Assessment Report, and in Chapter 6 of the 2007 IPCC Fourth Assessment Report which describes discussion of various possible reasons for the divergence which does not affect all the trees, and says that there is no consensus about the cause. It notes that Briffa et al. specifically excluded the post 1960 data, which is therefore not shown in the graph of their tree ring reconstruction in the AR4 report.
John Tierney wrote in The New York Times that "the graph adorned the cover of a report intended for policy makers and journalists. The nonexperts wouldn’t have realized that the scariest part of that graph — the recent temperatures soaring far above anything in the previous millennium — was based on a completely different measurement from the earlier portion. It looked like one smooth, continuous line leading straight upward to certain doom." Tierney quotes Michael E. Mann replying to a denier who raised this very issue about tree-ring data at the RealClimate blog, "No researchers in this field have ever, to our knowledge, ‘grafted the thermometer record onto’ any reconstruction. It is somewhat disappointing to find this specious claim appearing in this forum." Tierney continued, "Dr. Mann now tells me that he was unaware, when he wrote the response, that such grafting had in fact been done in the earlier cover chart, and I take him at his word. But I don’t see why the question was dismissed so readily, with the implication that only a tool of the fossil-fuel industry would raise it."
In December 2009, most scientists interviewed by The Philadelphia Inquirer considered there was ample other evidence supporting the original graph and they had not changed their views on the issues. In an exception, Richard Lindzen of the Massachusetts Institute of Technology alleged that the emails "explicitly refer to falsification and rigging of data" in covering up the divergence problem of tree ring proxies for the 1960s, which he said called earlier periods into question, and said "The trick here is replacing the kind of data you're using with something to make it look different.". Other climatologists disputed Lindzen's accusations. Thomas Peterson of the National Climatic Data Center said he had seen nothing in the emails that called the fundamental science into question, and Andrew Solow of Woods Hole Oceanographic Institution agreed that there was no trickery, saying he would use the word trick to describe some methodological step, but expressed the view that the basis of reconstructions had been unclear. Several scientific sources state that the decline being referred to is a decline in tree ring climate proxy metrics, not temperature. Andrew Watson, Royal Society Research Professor at the UEA, said that the scientists had drawn the line to follow the tree-ring reconstruction up to 1960 and the measured temperature after that."
Before the incident, Mann and others had presented reconstructions based on more proxies, and found similar results with or without the tree ring records.
The United States Environmental Protection Agency had issued an "endangerment finding" in 2009 in preparation for climate regulations on excessive greenhouse gases. Petitions to reconsider this citing the email were raised by the state of Texas and conservative activists and business groups the Coalition for Responsible Regulation, the Southeastern Legal Foundation, the Competitive Enterprise Institute and the coal company Peabody Energy. They said the email "evidences CRU staff’s effort to deliberately manipulate data to yield desired results", and highlighted the word "trick". The Southeastern Legal Foundation said it showed "an attempt to cook the books to conceal the fact that the famous ‘hockey stick’ is a manipulated, misleading barrel of scientific nonsense." The EPA rejected these allegations as both irrelevant and inaccurate, noting that the WMO cover illustration by Jones bore no relationship to the IPCC assessment reports, and used different methods. The issues with tree rings had not been hidden, but were extensively discussed in scientific literature and in IPCC reports. The EPA agreed with the UK Science & Technology Committee conclusion that "We are content that the phrases such as 'trick' or 'hiding the decline' were colloquial terms used in private e-mails, and the balance of evidence is that they were not part of a systematic attempt to mislead."

Peer review issue

In response to an e-mail mentioning a recent paper in the scientific journal Climate Research that questioned assertions that the 20th century was abnormally warm, Mann wrote in an e-mail of 11 March 2003:
Mann told the Wall Street Journal that he didn't feel there was anything wrong in saying "we shouldn't be publishing in a journal that's activist."
Mann was not alone in expressing concern about the peer review process of the journal. Half of the journal's editorial board, including editor-in-chief Hans von Storch, resigned in the wake of controversy surrounding the article's publication. The publisher later admitted that the paper's major findings could not "be concluded convincingly from the evidence provided in the paper. should have requested appropriate revisions of the manuscript prior to publication."
In an 18 December 2009 column in The Wall Street Journal, Pat Michaels alleged that pressure from Jones and Mann was responsible for the resignations at Climate Research. In a response also published in the WSJ, von Storch said that he left the post as chief editor of Climate Research "with no outside pressure, because of insufficient quality control on a bad paper – a skeptic's paper, at that." Michaels said the incident demonstrated how peer reviewed literature had been biased to prevent himself and others of like mind from publishing. In a response published by the WSJ, Mann said the only bias was for "well-reasoned writing that is buttressed by facts" and climate change deniers such as Richard Lindzen and John Christy had no problems with publishing their work in mainstream journals. He pointed to presidential science adviser John Holdren's 2003 statement that Michaels had "published little if anything of distinction in the professional literature, being noted rather for his shrill op-ed pieces and indiscriminate denunciations of virtually every finding of mainstream climate science."
The independent review commissioned by the University of East Anglia found that the strong reaction of the scientists to the Soon and Baliunas paper "was understandable, and did not amount to undue pressure on Climate Research."
The United States Environmental Protection Agency was petitioned about this issue by the Coalition for Responsible Regulation, the Ohio Coal Association, Peabody Energy, the Southeastern Legal Foundation, and the State of Texas The EPA concluded that the emails expressed displeasure but did not show that any action was in fact taken, and it is "expected and appropriate that researchers choose in which journals to publish, as well as recommend to their peers journals in which to publish or not publish. In this case, the bottom line is that the underlying science at issue has been shown to be flawed. The scientists’ actions were focused on this lack of scientific merit and the process that lead to it, and not an attempt to distort the science or the scientific literature." The EPA considered that "If anything, their actions aimed to police the peer review process and rectify a problem that threatened its scientific integrity."

Alleged exclusion of papers from IPCC report

In July 2004, referring to Climate Research having published a paper by "MM", thought to be Ross McKitrick and Pat Michaels, and another paper by Eugenia Kalnay and Ming Cai, Jones emailed his colleagues saying, "I can't see either of these papers being in the next IPCC report. Kevin and I will keep them out somehow – even if we have to redefine what the peer-review literature is!" At that time, Jones and Kevin E. Trenberth were lead authors on a chapter in the IPCC Fourth Assessment Report. Trenberth told the investigating journalist "I had no role in this whatsoever. I did not make and was not complicit in that statement of Phil's. I am a veteran of three other IPCC assessments. I am well aware that we do not keep any papers out, and none were kept out. We assessed everything we cannot possibly refer to all literature... Both of the papers referred to were in fact cited and discussed in the IPCC." He also made a statement agreed with Jones, that "AR4 was the first time Jones was on the writing team of an IPCC assessment. The comment was naive and sent before he understood the process." Jones could have been expected to be aware of the rules as he had been a contributing author for more than ten years, but this was his first time as a lead author with a responsibility for content of the complete chapter.
The IPCC has stated that its procedures mean there is "no possibility of exclusion of any contrarian views, if they have been published in established journals or other publications which are peer reviewed." Its chairman, Rajendra Pachauri, stated that the papers that had been criticised "were actually discussed in detail in chapter six of the Working Group I report of the AR4. Furthermore, articles from the journal Climate Research, which was also decried in the emails, have been cited 47 times in the Working Group I report."
A Nature editorial stated that the UEA scientists had been sharply critical of the quality of the two papers, but "neither they nor the IPCC suppressed anything: when the assessment report was published in 2007 it referenced and discussed both papers." Peter Kelemen, a professor of geochemistry at Columbia University's Department of Earth and Environmental Sciences, said that "If scientists attempted to exclude critics' peer-reviewed papers from IPCC reports, this was unethical in my view." Rajendra Pachauri responded that the IPCC has "a very transparent, a very comprehensive process which ensures that even if someone wants to leave out a piece of peer reviewed literature there is virtually no possibility of that happening."

Freedom of information

Several e-mails relating to freedom of information issues were the focus of controversy. In a 2 February 2005 e-mail, Phil Jones advised Michael Mann:
In another e-mail sent in May 2008, Jones asked Mann:
Before the emails were published, Jones had already announced in Nature News that he was working to release the raw data in a systematic way, and was writing to all the National Meteorological Organisations requesting their agreement to waive confidentiality. On 24 November 2009, four days after the start of the email controversy, the university stated that over 95% of the CRU climate data set had already been available for several years, and the remainder would be released when permissions were given.
Critics asserted that the e-mails showed that scientists were conspiring to delete e-mails and documents to prevent them from being released. George Monbiot, a supporter of the scientific consensus, wrote that Jones' resignation was warranted on the basis of his statement in this email alone, a statement Monbiot later retracted.
Pro-Vice Chancellor of Research at the University of East Anglia, Trevor Davies, said that no data were deleted or "otherwise dealt with in any fashion with the intent of preventing the disclosure". In response to allegations that CRU avoided obligations under the UK Freedom of Information Act, independent investigator Muir Russell plans to review CRU's "policies and practices regarding requests under the Freedom of Information Act".
The UK Information Commissioner's Office oversees the application of the Freedom of Information Act 2000. The Deputy Information Commissioner, Graham Smith, issued a statement which was published on 27 January 2010 and said that the e-mails showed that an FOI request made by David Holland, a climate sceptic from Northampton, had "not dealt with as they should have been under the legislation." Section 77 of the FOI Act prohibited public authorities from intentionally preventing the disclosure of restricted information, but it was too late to impose sanctions as there was a six-month statutory time limit. He was advising the university of East Anglia on its legal obligations, and the ICO would be considering whether to take regulatory action once reports of the independent and police investigations were available. The university said it had not been made aware of the statement by Smith.
In its submission to the Science and Technology Select Committee, the university denied allegations that it had refused to release raw data in breach of the FOIA. It subsequently released an exchange of correspondence with the Information Commissioner's Office to clarify that, in their opinion, the University had not been found in breach of any part of the FOI Act. UEA stated that the ICO had not completed its investigations and had not established any breach of the law, and that it had not sought any further evidence on the matter. It said that the ICO statement only referred to prima facie evidence, and that Mr. Holland's request at issue concerned private e-mails. The letter from the ICO had stated that "the prima facie evidence from the published e-mails indicate an attempt to defeat disclosure by deleting information. It is hard to imagine more cogent prima facie evidence. ... The fact that the elements of a section 77 offence may have been found here, but cannot be acted on because of the elapsed time, is a very serious matter." Evan Harris, a Liberal Democrat Member of Parliament, told The Times that it would be unwise for the university to attempt to portray the ICO's letter in a positive light, as the correspondence would be examined by the Committee. The UEA told the newspaper that the point being made in their submission was that "there has been no investigation so no decision, as was widely reported. The ICO read e-mails and came to assumptions but has not investigated or demonstrated any evidence that what may have been said in emails was actually carried out." The university said that the point being made in their submission was that "there has been no investigation so no decision, as was widely reported. The ICO read e-mails and came to assumptions but has not investigated or demonstrated any evidence that what may have been said in emails was actually carried out." The Science and Technology Select Committee report blamed the university for mishandling Freedom of Information requests and said it had "found ways to support the culture at CRU of resisting disclosure of information to climate change sceptics". The committee also criticised the ICO, and said that it made "a statement to the press that went beyond that which it could substantiate", but accepted that the six month statute of limitations restriction was insufficient, and said that this should be reviewed. It called for a full investigation by the Muir Russell inquiry or by the Information Commissioner, to resolve the question of whether there had been a breach of Section 77 of the FOIA.
In an e-mail of 12 November 2009, climate scientist Benjamin Santer of Lawrence Livermore National Laboratory commented on an FOI request for data and correspondence from climate sceptic blogger Stephen McIntyre:
In an Associated Press interview, McIntyre disagreed with his portrayal in emails, and said "Everything that I've done in this, I've done in good faith."
FactCheck stated that the great majority of CRU's data is already freely available, and the scientists were reluctant to supply their own correspondence, code and data to people whose motives seemed questionable to them. It is not clear that any deliberate obstruction happened, and emails show the scientists discussing with university officials and lawyers their obligations under the new legislation, informing critics that data is already freely available, or that the information has been sent to them. This question is to form part of the East Anglia investigation.
The University of East Anglia stated that the great majority of CRU climate data was already freely available, but the remainder was mainly owned by National Meteorological Services around the globe and subject to non-publication agreements. The Met Office was requesting new agreements to allow it to republish the raw data. CRU has a web page describing the progress that had previously been made in releasing this data, and giving details of non-publication agreements including the restrictions placed by the Met office on use of its data for bona fide research programmes. In a later BBC interview, Phil Jones said that the land station records developed at CRU show close agreement with the independent NCDC and GISS official records, which are based on raw data freely available from the Global Historical Climatology Network.
In response to the Met Office requests, some National Meteorological Services gave full or conditional agreement, others failed to respond, and the request was explicitly refused by Trinidad and Tobago and Poland. In discussions with the ICO about FOIA requests which had been made before the email controversy had begun, the university argued that the data was publicly available from the Met organisations, and the lack of agreement exempted the remaining data. In its decision released on 23 June 2011, the ICO stated that the data was not easily available and there was insufficient evidence that disclosure would have an adverse effect on international relations. The ICO required the university to release the data covered by the FOIA request within 35 calendar days. On 27 July 2011 CRU announced release of the raw instrumental data not already in the public domain, with the exception of Poland which was outside the area covered by the FOIA request. The data are available for download from Met Office website and from CRU. The university remained concerned "that the forced release of material from a source which has explicitly refused to give permission for release could have some damaging consequences for the UK in international research collaborations."

Gaps in understanding temperature variations

Critics also highlighted a passage in an e-mail sent by Kevin Trenberth on 12 October 2009 that discussed gaps in scientific understanding of recent temperature variations, in which Trenberth wrote:
Trenberth told the Associated Press that the email referred to an article he authored calling for improvement in measuring global warming to describe unusual data, such as rising sea surface temperatures. The word travesty refers to what Trenberth sees as an inadequate observing system that, were it more adequate, would be able to track the warming he believes is there.
In a statement on his NCAR webpage Trenberth states that,

Code and documentation

The CRU files also included programs written in Fortran, programmer comments and a readme file, which attracted considerable attention from programmers. The log covered more than four years work by a programmer, who had been frustrated by problems in relating climate data from numerous international sources. In a BBC Newsnight report, software engineer John Graham-Cumming said the code was "below the standard you'd expect in any commercial software" because it lacked clear documentation or an audit history. Graham-Cumming also reported finding a bug in the code's error handling which, if it occurred, would ignore data without warning.
Myles Allen, head of the Climate Dynamics group at the University of Oxford, said that the code investigated by Newsnight had nothing at all to do with the HadCRUT temperature record used for climate reconstructions, which is maintained at the Met Office and not at CRU. When he challenged Newsnight on this, they responded that "Our expert's opinion is that this is climate change code" and declined to retract the story. He commented that on the same basis "the quality of the code I use to put together problems for our physics undergraduates shows that we should not trust results from my colleagues who work on the Large Hadron Collider on the grounds that "it is all physics code"."
The United States Environmental Protection Agency investigation established that the programming notes recorded the process of updating the CRU TS2.1 dataset product to TS3.0. This product includes multiple climate variables, including temperatures, rainfall and cloud cover, and has nothing to do with the HadCRUT temperature record which uses the separate CRUTEM land temperature dataset. The CRU programmer had the difficult task of merging datasets at the same time as migrating code to a new computer system, finding ways of dealing with inconsistencies in data from disparate organisations, improving quality control and debugging.