Stress urinary incontinence (SUI) affects as many as one in five adult women (1), with a high socioeconomic burden. Patient reported outcome measures (PROMs) are increasingly used in clinical research for the assessment of the efficacy of interventions in pelvic floor disorders. QoL is a significant outcome domain of impact of SUI on women as well as of treatment efficacy. The aim of this study was to evaluate the content validity of the most commonly used and reported PROMs in published trials on SUI interventions, assessing their suitability for inclusion as core outcome and outcome measure sets (COS and COMS) in research and clinical practice.

Search terms were based on a previous systematic review (2) combined with an adjusted PROM and construct validity filter (3). We considered for inclusion PROMs in published randomised trials, which assessed QoL. 

A literature search was conducted using EMBASE, MEDLINE and PsycINFO databases. The search filter used in PubMed, was modified for other databases (3). Hand searches on Google scholar were additionally undertaken.

To assess content validity, any report regarding the development of the identified PROMs was included, irrespective of the published format. Studies were eligible if the population included adult women (18 years or older), with SUI. Studies had to be conducted on humans, available as full-text articles, and published in English, or with an available English translation. Studies concerning cross-cultural adaptation and translation of a PROM were included as content validity studies if they performed a pilot study to evaluate cross-cultural comprehensibility. 

To evaluate PROM development 35 standards were assessed across two domains; the initial domain involving general design requirements for the PROM, subsequently followed by the assessment of the quality of the cognitive interview study. Each standard was rated using the four-point scale as either: “inadequate”, “doubtful”, “adequate”, or “very good”. 

To assess the quality of the content validity studies, a further set of COSMIN (www.cosmin.nl) guidelines were used. Thirty-one standards reported relevance, comprehensiveness, and comprehensibility of the PROM by professionals and patients. Again, using the four-point scale. 

Total scores for each domain were calculated by using the ‘worst score counts’ method (using the lowest grade of any standard in that box). 

Evidence synthesis comprised of two steps. The PROM development studies, content validity studies, and the content of the PROM, were rated against the 10 established criteria of good content validity.

Quality of evidence of the PROM development studies, was then assessed by applying a modified GRADE (Grading of Recommendations Assessment Development and Evaluation) approach.

We selected the seven most commonly used and reported PROMS in randomised trials on SUI interventions in female populations. We evaluated the Urinary Distress Inventory (UDI), Urinary Distress Inventory 6 (UDI-6), International Consultation of Incontinence Questionnaire – Short Form (ICIQ-SF), International Consultation of Incontinence Questionnaire – bladder diary (ICIQ-Bladder Diary), Incontinence Impact Questionnaire (IIQ), Incontinence Impact Questionnaire-7(IIQ-7), and Patient Global Impression of Improvement (PGI-I). Only one PROM (PGI-I), was developed in a population with diagnosed stress urinary incontinence. Of the remaining PROMS, 4 (UDI, IIQ, UDI-6, IIQ-7) were designed to specifically assess QoL in women with urinary incontinence, and the ICIQ-SF to assess QoL in women as well as men. 

The overall quality of PROM design was deemed inadequate for 5 PROMs (UDI, IIQ, UDI-6, IIQ-7, and PGI-I). This was due to the lack of an appropriate interview guide, and no obvious recordings of interviews, meaning concept elicitation was scored as inadequate. Despite this, 4 (IIQ, UDI, ICIQ-SF, ICIQ Bladder Diary) of the 7 PROMs assessed did involve patients in concept elicitation. 

Of the 7 PROMs, only 2 (ICIQ-SF, ICIQ-Bladder diary) had cognitive interviews in their development process. The comprehensibility of the ICIQ-SF was deemed doubtful, as it was unclear if guides were used to conduct interviews, and if these were in turn transcribed or recorded. With regards to comprehensiveness of ICIQ-SF, this too was scored as doubtful, again due to the lack of clarity surrounding the interview process, and adaptation of the PROM. 

Comparatively, the ICIQ-Bladder Diary scored adequately in terms of comprehensibility, the main attributing factor being the inclusion of interview recordings. An adequate rating was also given for comprehensiveness. The authors documented problems faced by patients when answering the questionnaire, and appropriately addressed these issues (Table 1). 

We identified one study evaluating content validity of a single PROM – ICIQ Bladder Diary. The study assessed relevance, comprehensibility, and comprehensiveness. All 5 domains were classified as doubtful, as it was unclear if interviews were based on an appropriate topic or guide, or if an appropriate approach was used to analyse the data. 

No high-quality evidence was available for the 7 PROMs included in this review (Table 2). Only the ICIQ- Bladder diary, was deemed moderate across the board, suggesting this PROM likely has the best content validity for QoL in women with stress urinary incontinence.

Overall, there is a clear lack of content validity studies, with our searches returning only one content validity paper, assessing relevance, comprehensiveness, and comprehensibility of one PROM (ICIQ-Bladder Diary). 

Development should be undertaken in a population reflective of the population of interest, to ensure relevance and comprehensiveness. Only 3 out of 7 PROMs used in this study involved patients in the development process. Comparatively, only 2 out of 7 PROMs conducted cognitive interviews suggesting that information regarding content modification was not extensively considered. Patients were not given an opportunity to report problems with relevance or comprehensibility of the measurement tool, meaning data may have been incorrectly recorded or missed altogether. 

Despite recommendations existing to aid development of PROMs with high content validity, our findings concur with other literature highlighting the issue with poor reporting of qualitative research regarding content validity. Guidelines suggest that as a minimum development of a PROM should include: 1) a literature review, 2) concept elicitation reviews or the use of focus groups, 3) data analysis, 4) item generation, and 5) performing cognitive interviews. In 2016 COSMIN methodology provided an approach to assess content validity, prior to this a clear framework was missing, leading to a lack of standardisation of development. Studies previously published have not assessed content validity. 

This study was completed as part of a wider project to establish COS and COMS in research and clinical practice for women with SUI. COS/COMS aims to reduce the variation of outcome reporting in clinical trials. Inconsistencies within outcome reporting in women with SUI have been highlighted by previous systematic reviews. COSMIN and COMET guidelines outline a methodological approach to aid developers when producing a COS.

This review has highlighted gaps in the quality of evidence surrounding content validity for PROMs used to assess QoL in women with SUI. To collect clinically relevant data from patients and ensure we can record QoL in a meaningful way, it is imperative that development of PROMs is based on robust methodological principles.

Nikolopoulos, K.I., Betschart, C. and Doumouchtsis, S.K. (2015) ‘The surgical management of recurrent stress urinary incontinence: a systematic review’, Acta Obstetricia et Gynecologica Scandinavica, 94(6), pp. 568–576. doi:10.1111/aogs.12625.
Loganathan, J., Coffey, J., Doumouchtsis, S.K., and chorus: An International Collaboration for Harmonising Outcomes, Research and Standards in Urogynaecology and Women’s Health.  (2022). ‘Which patient reported outcomes (PROs) and patient reported outcome measures (PROMs) do researchers select in stress urinary incontinence surgical trials? – a systematic review’. International Urogynecology Journal. Doi: https://doi.org/10.1007/s00192-022-05123-7
Prinsen, C. A., Mokkink, L. B., Bouter, L. M., Alonso, J., Patrick, D. L., De Vet, H. C., et al. (2018) 'COSMIN guideline for systematic reviews of patient-reported outcome measures', Quality of Life Research, 27, pp. 1147-1157. DOI: 10.1007/s11136-018-1798-3

Continence 2S2 (2022) 100347
DOI: 10.1016/j.cont.2022.100347