June 9, 2026
In 2025, a systematic review assessed the methodological rigour of 76 clinical practice guidelines published by the European Society of Cardiology (ESC) and the American College of Cardiology/American Heart Association (ACC/AHA) between 2013 and 2024. These are not obscure documents from minor societies. They are among the most cited, most influential guidelines in medicine, the recommendations that cardiologists worldwide rely on to make treatment decisions for countless patients.
Using the AGREE II instrument, which evaluates whether guideline recommendations are developed using systematic, transparent, and reproducible methods, the average rigour of development score was 29 out of 56 possible points, just over half of the maximum score. Forty-six percent of all recommendations were based solely on expert opinion. The lowest-scoring areas were systematic search of evidence, use of predefined criteria for selecting evidence, and external review. Scores did not improve across the entire 12-year study period.1
Read that again. The guidelines that drive cardiology practice globally, published in the most prestigious journals, authored by the most distinguished panels, scored just over half marks on the quality domain that measures whether the recommendations were developed using systematic, transparent, and reproducible methods. And that number hasn’t budged in over a decade.
If the world’s leading specialty societies are struggling with this, the problem is not competence. It is that the methods section is treated as an afterthought, administrative boilerplate to be written after the real work is done. That assumption is where most guidelines fail.
The methods section of a guideline is not a formality. It is the contract between the development group and the reader. It tells the reader how the evidence was found, how it was appraised, how the recommendations were formulated, and how conflicts of interest were managed. When the methods section is vague, incomplete, or inconsistent with the rest of the document, the recommendations it supports become difficult to trust, regardless of how distinguished the panel or how clinically sensible the guidance.
The AGREE II instrument dedicates an entire domain: Domain 3, Rigour of Development to evaluating exactly these elements. It asks whether systematic methods were used to search for evidence, whether the criteria for selecting evidence are clearly described, whether the methods for formulating recommendations are transparent, whether the health benefits and risks were considered, whether there is an explicit link between recommendations and supporting evidence, whether the guideline was externally reviewed, and whether a procedure for updating is provided.2
Understanding what this domain evaluates during development, rather than discovering it at peer review, is the difference between a methods section that survives scrutiny and one that triggers a major revision request.
A guideline’s evidence base is only as strong as the search that produced it. The methods section should specify the databases searched (commonly MEDLINE/PubMed, Embase, and the Cochrane Library), the date range, the search terms or strategy, and any language or study design restrictions. Ideally, the full search strategy should be available as a supplementary appendix.2–4
The distinction between a systematic and a narrative literature review matters here. A systematic review uses predefined eligibility criteria, reproducible search methods, and transparent reporting, often documented using a PRISMA flow diagram. A narrative review is more selective and less reproducible but may be appropriate for consensus-driven documents where the clinical questions are broad and the evidence base is heterogeneous.3,4
What the methods section should never do is claim one approach while describing another. Stating that a “systematic literature review was conducted” when the search was limited to PubMed, used a handful of MeSH terms, and screened references informally is a misrepresentation that peer reviewers will identify. Honesty about what was actually done for instance “a structured narrative review was conducted” is not an admission of weakness, it’s methodological integrity.
If evidence was formally appraised, the methods section should describe the framework used (most commonly GRADE), how evidence quality was assessed for each outcome, and how the assessment informed recommendation strength. GRADE is built around evidence profiles or Summary of Findings tables, and these should be included as supplementary materials.5,6
If formal evidence appraisal was not conducted, which is entirely appropriate for many consensus-driven documents, this should be clearly stated, and the methods section should explain how recommendation confidence is conveyed. A single sentence can accomplish this: “Stronger directive language (e.g., ‘should,’ ‘is recommended’) indicates high panel consensus supported by consistent data, whereas more conditional phrasing (e.g., ‘may be considered’) reflects limited or conflicting evidence or greater reliance on expert opinion.”
This kind of transparency is not a concession. It is an acknowledgement that different document types warrant different methodological approaches, and the reader deserves to know which one was taken. As we discussed in Part 2 of this series, using GRADE language without the GRADE process creates a confidence signal the methodology does not support, and it is one of the most common errors identified by peer review.
The methods section should describe how the panel moved from evidence, or expert opinion, to recommendations. This includes the consensus methodology used (Delphi, modified Delphi, nominal group technique), the number of rounds, the predefined threshold for agreement, and how disagreements or non-consensus items were handled.7
For guidelines using GRADE, the Evidence-to-Decision (EtD) framework provides a structured process for this transition, incorporating evidence quality, the balance between benefits and harms, patient values and preferences, resource use, health equity, acceptability, and feasibility.8 Not every guideline uses EtD formally, but the considerations it addresses should be reflected in the development process.
A methods section that states “recommendations were formulated through panel discussion” without specifying the process is insufficient. As we explored in Part 3 of this series, formal consensus methods such as Delphi were developed to reduce the influence of dominant voices and make expert judgment more structured and transparent. The method matters, and the methods section needs to describe it.
The IOM standards require that conflicts of interest: financial, intellectual, and institutional, be declared by all panel members, that the majority of the panel be free of relevant conflicts, and that the chair or co-chairs be unconflicted. Disclosure is necessary but not sufficient; management requires action, such as recusing conflicted members from discussions and votes on recommendations where their conflicts are relevant.3
The methods section should describe how conflicts were identified, how they were evaluated, and what management strategies were applied. A blanket statement that “all authors completed ICMJE disclosure forms” addresses disclosure but says nothing about management. More broadly, disclosure alone does not demonstrate that conflicts were actively managed.
Both the IOM and AGREE II call for external peer review of the guideline prior to publication, that is review by individuals who were not part of the development panel.2,3 The methods section should indicate whether external review was conducted, who the reviewers were in general terms, and how their comments were addressed.
This step is frequently omitted. Its absence is one of the most common areas of low scoring in AGREE II appraisals, and it is one of the easiest to address if planned from the outset.
A credible guideline should include a stated plan for when and how it will be updated. Evidence evolves, and recommendations that were appropriate at the time of development may become outdated within a few years, or in some fields within a few months. The IOM standards include this as one of eight required elements, yet many published guidelines omit it entirely.3
The plan need not be elaborate. A statement such as “This guideline will be reviewed for currency within five years of publication, or sooner if new evidence emerges that materially affects the recommendations” is sufficient.
Perhaps the most damaging methodological failure is writing the methods section after the recommendations have been finalised, reverse-engineering the stated methodology to match what was actually done. This produces inconsistencies that manifest throughout the document: GRADE ratings without evidence profiles, a “systematic review” supported by a thin reference list, or consensus methodology described but never implemented as described.
The remedy is straightforward: develop the methods protocol before the guideline development process begins. The protocol should specify the search strategy, the evidence appraisal framework, the consensus methodology, the agreement threshold, and the conflict-of-interest management plan. This document then becomes the basis for the methods section, ensuring consistency between what was planned, what was done, and what is reported.
The cardiology guidelines that scored 29 out of 56 on Rigour of Development were not written by unqualified panels. They were written by some of the best clinicians and researchers in the world. The problem was never the expertise, it was the process. A methods section written prospectively, grounded in a protocol developed before the first recommendation was drafted, would have changed those scores. It would likely have improved methodological transparency and strengthened confidence in the recommendations, because a rigorous process does not just look more credible, it is more credible.
Stay tuned for our fifth and final installment of this series, where examine the avoidable mistakes that can weaken guideline development, compromise trust in the final recommendations and unravel even well-intentioned guideline projects.
References