Developing a Core Outcome Set for Cytoreductive Surgery for Colorectal Cancer with Peritoneal Metastases: A Mixed-Method Study Protocol

Abstract

As the number of centres offering cytoreductive surgery (CRS) for colorectal cancer with peritoneal metastases (CPMs) is increasing worldwide, research is focused on establishing better patient selection and ensuring that new techniques have positive impacts on survival. However, high-impact comparative research in this field is limited by the heterogeneity of outcome measurement and reporting. Additionally, as there are comparatively few randomised controlled trials reporting comprehensive patient-reported outcomes, it is possible that key stakeholders such as patients and carers are underrepresented in the current literature. A core outcome set (COS) for CRS with or without intraperitoneal chemotherapy for the treatment of CPMs, supported by clinicians and patients, will promote homogenous comparison across trials and optimise the utility of research findings. We have established a comprehensive protocol based on the Core Outcome Measures in Effectiveness Trials (COMETs) method to facilitate this. A systematic review will identify all the outcomes reported in the literature, whereas a semi-structured interview will identify outcomes considered important by patients and carers. The identified outcomes will populate an international Delhi survey, distributed to patients, carers, surgeons, oncologists, nurses, and allied health clinicians. Outcomes reaching international consensus of importance will be further discussed in a face-to-face workshop between patients, carers, and clinicians. This process will inform the development of a final COS for CRS for patients with CPMs.

1. Introduction

Cytoreductive surgery (CRS) with or without intraperitoneal chemotherapy (IPC) provides a chance of survival for selective patients with colorectal cancer with colorectal peritoneal metastases (CPMs). The peritoneum represents the third most common site of colorectal cancer metastases, and 8.3% of patients with colorectal cancer will develop CPMs [1,2]. Historically, this was considered distant metastatic spread, with the mainstay of treatment being systemic palliative therapies. Survival rates without surgical intervention have been reported to be as low as 5–9 months [3,4]. In contemporary practice, CPMs are considered a potentially curative locoregional disease. Advances in chemotherapy agents and refined CRS techniques have demonstrated the efficacy of CRS and IPC to improve survival in this group of patients in both curative and palliative settings [5].

As outcomes have improved, there has been a corresponding increase in clinical interest and academic work. However, recent systemic reviews highlight that within this field, trials have significant variability in surgical methods and reported outcomes [6,7,8,9]. This heterogeneity limits the meaningful synthesis and comparison of data between trials and hinders the development of high-level evidence for CPMs treatment. Currently, the most commonly reported outcomes include postoperative morbidity and survival, with a distinct disparity in the reporting of quality-of-life factors and other patient-reported outcome measures. Amongst commonly reported outcomes, e.g., morbidity, definitions are not always consistent or well defined, further limiting comparative interpretation and analysis [7,10].

Although CRS offers selected patients a chance of cure, it is associated with significant morbidity. Therefore, patients are carefully selected by multidisciplinary teams with theoretically shared decision making between clinicians and patients [4]. Additionally, as new CRS centres are established globally, there is a need for a periodic evaluation and assessment of outcomes to ensure treatment quality [11]. Therefore, it is critical that clinical trials and other research investigating CRS for CPMs report relevant outcomes, supported by patients and clinicians, so that complex treatment decisions are made and evaluated with the highest-quality evidence base.

A core outcome set (COS) “is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific areas of health or health care”, which aims to reduce heterogenicity in data reporting, reduce selective reporting, and improve the clinical relevance of trials [12]. The Core Outcome Measures in Effectiveness Trials (COMETs) initiative provides guidance and best practice in developing rigorous core outcomes. This protocol considers the Core Outcome Set-STAndardised Protocol (COS-STAP) statement [13] in its design to achieve the following objectives:

• Identify the outcomes of CRS (with or without IPC) for CPMs that are currently reported in the literature.
• Identify the outcomes of CRS for CPMs which are considered important by patients and their carers.
• Gain consensus from an international group of multidisciplinary clinical experts, patients, and carers around the most important outcomes of CRS for CPMs.

2. Study Design and Methodology

This study will employ a mixed-methods study design, conducted in accordance with recommendations from the Core Outcome Measures in Effectiveness Trials (COMETs) initiative [14]. This study will be conducted across three distinct phases.

• Phase 1: Information gathering.

  ○

  A comprehensive systemic review of the up-to-date literature surrounding CRS for CPMs to identify all the reported outcomes.

  ○

  Semi-structured interviews with patients who have undergone CRS for CPMs and their carers to identify which outcomes are considered most important following surgery.

• Phase 2: Employing Delphi methodology, a series of iterative anonymous surveys will be disseminated to key stakeholders (patients, clinicians, carers) to achieve an international consensus on the importance of identified outcomes.
• Phase 3: Refining consensus. The list of outcomes identified in phase 2 will be discussed in a face-to-face/teleconferenced meeting between a representative sample of key stakeholders.

2.1. Phase 1: Information Gathering—Systematic Review

A comprehensive search strategy will be conducted in Medline, Embase, Scopus, Cochrane Central Register of Controlled Trials (EBM reviews), and the Cumulative Index to Nursing and Allied Health Literature (CINAHL) to identify all the reported outcomes of CRS for CPMs. The full protocol, selection criteria, and search strategy have been prospectively registered on the PROSPERO database (PROSPERO ID CRD42023411541).

Studies in the literature published after the year 2000 written in the English language, pertaining to CRS, with or without IPC, for adult patients (>18) with CPMs will be included. For this review, intraperitoneal chemotherapy will include Heated Intraperitoneal Chemotherapy (HIPEC), Early Postoperative Intraperitoneal Chemotherapy (EPIC), or Sequential Postoperative Intraperitoneal Chemotherapy (SPIC). Studies with a cohort of <20 people, case–control studies, and case reports will not be included; in studies where CRS was indicated for various pathologies, only cohorts in which 90% of patients had colorectal cancer will be included.

Two researchers will independently review titles and abstracts against the eligibility criteria to determine potential suitability for inclusion. Full-text articles will be retrieved and screened for final eligibility and data extraction. The number of articles identified and excluded will be recorded at each stage, and a rationale for exclusion will be recorded. At any stage, disagreements between the involved researchers will be discussed with a third senior member of the research team.

Data extraction will include all the reported outcomes, including the tools used to measure them, definitions, and time points. These results will be categorised into key domains, as defined by the COMETs initiative [15], and sufficiently similar outcomes will be collapsed together at the discretion of the project supervisory group.

2.2. Phase 1: Information Gathering—Semi-Structured Patient Interviews

At this stage, an exploratory qualitative design will be used to determine important outcomes of CRS for CPMs accordingly to patients and carers. A 30 min semi-structured interview will be conducted with a focus on participant-led dialogue. The target sample for this phase will be 10–15 patients.

• Participant Sampling

Participants undergoing CRS for CPMs at the Royal Prince Alfred Hospital, between April 2017 and December 2023, will be contacted if they meet the following eligibility criteria:

• Inclusion Criteria

• Adults > 18 years of age.
• Patients who have undergone cytoreductive surgery with or without IPC for colorectal malignancy.
• Patients who have had surgery >6 months ago.
• Patients who have the capacity to provide informed consent.
• Patients able to participate in the interview in the English language.
• Patients who are well enough to participate in the interview as deemed by a primary treating clinician.

• Exclusion Criteria

• Patients who underwent treatment with purely palliative intent.
• Patients who have undergone CRS for other malignancies, e.g., ovarian or indications.

Following a final death check, the identified individuals will be sent an official invitation to participate, a participant information sheet, and a consent form. Consent may be provided through a physical form or through an online eConsent platform; consent will be confirmed verbally prior to the commencement of the interviews. It will be stressed that participation or lack thereof will have no impact on the participant’s current treatment, relationships with their treating clinician(s), or any other organisation affiliated with this study.

Purposive sampling will be used to maintain a representative sample and encourage diversity when selecting the study cohort. Characteristics including age, location of residence, use of IPC in treatment course, and presence of a stoma will be used to guide recruitment as per the sampling matrix (

Table 1. Semi-structured patient interviews sampling matrix.

Matrix Criteria	Target Participant Number
Age At Surgery
18–40	2–3
41–65	7–9
>65	2–3
Gender
Male	5–8
Female	5–8
Place of Primary Residence
Metropolitan	7–8
Rural/Regional	3–5
Stoma
Present	7–9
Not Present	7–9
Target Total	10–15

“Target Total” refers to the total number of participants in the study, not the sum of individual criteria, as many participants will fall into multiple categories, e.g., a 60-year-old male from rural origin.

).

Patients’ demographic information and treatment data will be extracted from hospital electronic medical records, and missing data will be requested from patients at the time of the interview. Data obtained from medical records will include participant age, age at time of surgery, gender, location of principle residence, diagnosis, surgical data, preoperative treatments, treatment complications, hospital length of stay, re-admissions, and postoperative adjuvant treatments (e.g., chemotherapy).

Participant interviews will use a semi-structured format to facilitate the discussion of postoperative experiences, observations, and important outcomes. Open-ended questions will be used to guide discussion bolstered by additional prompts from an interview guide. The interview guide may be modified in accordance with the results of the systematic review (phase 1), or iteratively based on previous interviews to ensure relevance and completeness of the discussion. Upon concluding the interview, patients will be asked if their carers would consider participation in this study. The carers identified will be contacted, and the interviews will be conducted in a similar fashion, with the goal of performing approximately five carer interviews.

Interviews will be conducted online or in person, and audio will be recorded and transcribed in full. Identifying data will not be recorded, and the recordings will be stored against a unique patient identifier to protect patient identity. Transcriptions will be imported into NVivo qualitative analysis software (Nvivo 11, QSR International, Burlington, MA, USA). Thematic analysis, as guided by COMETs taxonomy, will be used to extract and categorise identified outcomes. This analysis will be discussed by senior advisory members of the research team, considering the frequency of each reported outcome and its justification.

3. Phase 2: Achieving Consensus Amongst Stakeholders Using Delphi Methodology

Following phase 1, principal study investigators will meet to discuss the comprehensive list of research outcomes identified within the literature and via patient interviews. Each outcome will be considered and merged with sufficiently similar outcomes and grouped into thematic domains. Additionally, this long list will be checked against existing core outcome sets for colorectal cancer [16,17] for completeness. Any outcomes found in these sets, but not identified in phase 1 of this study, will be discussed and considered for inclusion. All outcomes will be attached to a short description where applicable. Outcomes which appear sparsely in the literature, determined to be of little importance by our expert group and not identified in the patient interviews, will be discarded. These outcomes will be converted into clinical and plain language versions. The validity and readability of these plain language versions will be trialled with a third-party group, consisting of a patient and clinician. The wording and syntax of each outcome will be adjusted accordingly.

The final outcomes will be presented to stakeholder groups (clinicians, patients, carers) across two iterative survey rounds through a secure web application, REDCap. The aim of this phase is to refine the initial long list of identified outcomes and generate a final list of important reportable outcomes for CRS through stakeholder consensus.

• Participants

Participants will be recruited from three main stakeholder groups, namely clinicians, patients, and carers. All participants must be >18 years of age to partake in the survey and be able to complete the survey in the English Language. Participants will be sent an email containing an official invitation to participate and information sheet. Consent will be implied by participation in the online survey, as will be outlined in the supplied documentation, and may be freely withdrawn throughout the study.

• Clinicians with experience in CRS for colorectal cancer, including peritoneal malignancy surgeons, surgical oncologists, colorectal surgeons, medical oncologists, clinical nurse specialists, and allied health professionals, will be identified through the following:

  ○

  Australia and New Zealand cytoreductive surgery multidisciplinary teams (MDTs). MDT co-coordinators at the Royal Prince Alfred Hospital, St George Hospital, Peter McCallum Cancer Centre, Mater Hospital Brisbane, will be contacted to help distribute study invitations to their clinical mailing list.

  ○

  Members of the Peritoneal Surface Oncology Group International (PSOGI) collaborative.

• Patients who have undergone CRS for CPMs at the Royal Prince Alfred Hospital between April 2017 and December 2023 will be contacted for participation. The MDT co-coordinators of national and international centres will also be encouraged to distribute recruitment letters to eligible patients. Interested patients will be contacted by the research team with a link (via email) or letter to participate.
• Patients who participate will be asked to forward an invitation email to their carers, who will be contacted, if interested, in a similar way.

Participants will be contacted via email, or via letter if preferred, and each survey round will be open for 30 days, with reminder emails sent at 10 and 20 days. All participants will be encouraged to pass on the details of the study and initial invitation to their own contacts/networks who meet the eligibility criteria to enhance the sample size and study reach (snowball sampling).

• Delphi Methodology

Delphi First Round: Participants will identify which stakeholder group they belong to; they will also be assigned a unique study ID to deidentify data and anonymise responses. The first round will contain two sections. Section 1 will collect demographic data from the participants.

• Patients: Age, gender, time since surgery, presence of stoma, type of IPC administered (if relevant), metropolitan or rural/regional residence, and interest in taking part in an outcome setting workshop for CRS and CPMs.
• Carers: age, gender, and relationship to patient.
• Clinicians: age group, gender, specialty, highest level of education, number of years’ experience in treating patients with CPMs, number of procedures performed annually (in the case of surgeons), country of practice, and interest in taking part in an outcome setting workshop for CRS for patients with CPMs. Clinicians will include all members of the multidisciplinary team, including surgeons, medical oncologists, specialist nurses, stoma nurses, physiotherapists, and dieticians.

In Section 2, participants will be presented with the list of outcomes identified in phase 1 of this study. Outcomes will be grouped according to their domain, so similar outcomes will be reviewed in unison. Each outcome will be described in scientific terms (for clinicians) and in plain language (for patients). Participants will be asked to rate each outcome on a nine-item Likert scale grouped into three categories: 1–3 (limited importance); 4–6 (important but not critical); and 7–9 (critically important). A concluding open-ended question will ask participants to identify and list any additional outcomes they feel were not included.

Although there is no defined minimum participant number to obtain consensus in a given Delphi study, we aim to have 150–200 participants at this stage.

While there is no universally agreed upon definition for consensus in Delphi studies, we selected definitions based on recent systematic reviews [18]. Consensus within a particular stakeholder group will be defined as:

• Consensus in: 70% or more respondents within a stakeholder group rate the outcome as critically important (7–9) and fewer than 15% of respondents rate the outcome as limited importance (1–3).
• Consensus out: 70% or more respondents within a stakeholder group rate the outcome as carrying limited importance (1–3) and fewer than 15% of respondents rate the outcome as critically important (7–9).
• No Consensus: Neither of the above definitions are met.

Delphi Second Round: In this round, all outcomes which reached definitions of consensus in or no consensus will be presented to participants; outcomes reaching the definition of consensus out will be discarded. Participants will be asked to rate each item on the same nine-item Likert scale as round 1, with similar grading. They will be provided with feedback from round 1 with their previous score for each outcome, as well as the median score from their respective stakeholder group. Participants will be requested to reflect on the information provided as they score each outcome on this focused list again. An open-ended question will again be posed, allowing for participants to provide feedback on the listed outcomes and identify any outcomes they feel were missed.

Research will review the responses to open-ended questions from each round and discuss the validity of the identified outcomes for inclusion.

It is expected that not all participants who complete survey round 1 will also complete survey round 2 (dropout), although the importance of this will be stressed in the attached information sheet during the study introduction. In round 1, participants will be provided with a unique identifier, allowing for attrition between rounds to be measured. This will also allow for an analysis to be conducted between those who completed and those who partially completed the Delphi.

• Statistical Analysis

Descriptive statistics will be used to characterise the sample; for the results of the Delphi study, median scores along with interquartile ranges will be used to summarise and analyse responses in conjunction with definitions for consensus [19].

4. Phase 3: Refining Consensus through a Core Outcomes Workshop

During this phase, a face-to-face meeting will be held amongst a group of key stakeholders to discuss outcomes of the Delphi process and develop the final COS. The objective will be to clarify outcomes for which there was agreement and discuss those which did not reach consensus during the Delphi process.

Recruitment and Participants

We will aim to have a working group of 15–20 participants, representing key stakeholder groups, including surgeons, medical oncologists, allied health, nursing, patients, and carers. Participants will be identified from those expressing interest during the Delphi Study and local CRS MDT networks. The following sampling matrix will be used to guide purposive sampling for this phase to ensure the adequate representation of stakeholder groups (

Table 2. Core outcomes workshop sampling matrix.

Key Demographic	Target Participant Number
Clinicians
Surgeons	8–10
Medical oncologists	2–4
Allied health and nursing	2–3
Patients
Male	1–2
Female	1–2
Carers	1–2
Overall Sample	15–20

).

Meeting Setting

On the allocated date, the workshop will take place at a designated meeting room within the Royal Prince Alfred Hospital Institute of Academic Surgery. We estimate that the workshop will be conducted over 1–2 h. Participation will also be offered remotely via teleconference software. Participants will verbally confirm their consent prior to meeting commencement and be asked to sign a consent form. Similarly, as in phase 1, participants will be able to withdraw their consent/participation at any time, and patients will be informed that withdrawal will have no impact on their ongoing care or relationship with study affiliates.

Meeting Format

The meeting will be hosted by two investigators who will use a template to structure the discussion. Meeting minutes will be recorded by a nominated typist using word-processing software.

The session will commence with an introduction to COS, followed by a review of both Delphi rounds. Initially, we will propose that outcomes with ‘consensus in’ across three stakeholder groups in Delphi round 2 be included in the final COS. Conversely, outcomes labelled as “consensus out” across three groups will be excluded. Participants will anonymously vote “yes” or “no” using electronic keypads to either accept the proposal or suggest outcomes requiring further discussion. All remaining outcomes (i.e., those categorised as “consensus in” or “consensus out” by only one or two stakeholder groups or ‘no-consensus’) will then be deliberated and voted upon.

Consensus on a particular outcome will be determined when less than 30% of participants disagree on its inclusion. The final COS will undergo review and discussion by the meeting participants.

5. Ethics Approval

This study has received ethical approval from the ethics committee of Sydney Local Health District (Protocol No. X23-0318 & 2023/ETH01852 on 15 September 2023) and received site-specific authorisation on 3 October 2023.

6. Discussion and Dissemination

CRS continues to be a growing field, and its safety, application, and evaluation should be confirmed through high-level evidence. The development of an internationally applicable COS which is supported by clinicians, patients, and carers will allow for standardised high-quality evidence to inform the decisions of health care practitioners, patients, and policy makers.

The final COS and results from this project will be published in peer-reviewed journals and be presented at relevant conferences and seminars. All published work will follow the Core Outcome set-STAndards for Reporting (COSTAR statement), as recommended by the COMETs initiative [20]. Results will be available to all study participants, as well as disseminated on official social media pages. Plain language summaries of the study results will also be disseminated to the patients involved.