Development of PatientCentered Questionnaires FDAIndustry Workshop Washington DC

Development of Patient-Centered Questionnaires FDA/Industry Workshop – Washington DC 2005 Cindy Rodenberg, Ph. D Procter & Gamble Pharmaceuticals

Why Develop A New Instrument? v Development of clinical treatments requires a suitable instrument to measure aspects particular to the disease and population. v Previously published instruments may: Ø Not adequately measure intended treatment effect – e. g. new research area Ø Not adequately reflect the patient population Ø Use language that is dated Ø Not translated or linguistically harmonized 2

Population-specific: Is that really important? 3

Three Steps of Instrument Development Preliminary Instrument Development Face and content validity Qualitative Content Validation Translation and linguistic validity Quantitative Analysis And Validation Reliable, valid, and sensitive to treatment effect 4

Step 1: Preliminary Instrument Development v Item Generation - Identify characteristics of the disease and ensure adequate sampling of content to provide valid measurement Ø Ø Face validity – Appears relevant to the outcome intended to capture. Content validity – Sampling of items, presentation, and measurement of all aspects of the disease or states relevant to the patient. 5

Importance of content validity v Study specific instruments may be biased “…if since the sponsor decides what it wants to ask, and thus can emphasize areas where the product should excel and deemphasize potentially troublesome areas…” v Smith, N. Quality of Life Studies from the Perspective of an FDA Reviewing Statistician, Drug Information Journal, 1993; 27: 617 -623 6

Methods – Item Generation v Experts Ø Menopausal Sexual Interest Questionnaire (MSIQ) – 10 item questionnaire judged to capture key components of sexual desire and response v Literature review/Existing questionnaires Ø Mini Mental from the WAIS Intelligence Test v Patient-centered approach - Focus groups and individual patient interviews Ø Profile of Female Sexual Function (PFSF) 7

Patient Centered Approach v Focus groups and individual interviews Ø Structured, Semi-structured, or Unstructured v Subjects selected to ensure adequate representation across relevant population subgroups v Sample until ‘data saturation’ – in other words, no new information. Ø Approximately 5 -10 subjects per subgroup 8

Response Levels v Direct Estimation Method – Directly quantify magnitude of a trait Ø Visual Analog Scale: Arthritic Pain? Worst pain ever Ø No Pain Adjectival Scale: How would you rate your overall response to treatment? Poor Fair Good Excellent 1 2 3 4 9

Step 2: Qualitative Content Validation v Elimination and refinement to ensure items: Ø Represent patients’ symptoms/experiences Ø Have clear and unitary meaning Ø Have similar meaning across translations v Cognitive Interview Technique Ø Ø Ø 1 -on-1 Interviewer probes on thought process used by patient in determining response Iterative process 10

PFSF Content Validation I feel like a sexual person French: I feel like a prostitute I felt relaxed about sex Relaxed about the subject of sex – not actually having sex Vocabulary too high I felt apathetic about sex It took forever to get aroused Patients who didn’t have sex over the past month responded “never” 11

Step 3: Quantitative Statistical Approaches v Methods for item reduction and domain identification v Methods for assessing reliability and validity 12

Item Reduction and Domain Identification v Data quality assessment Ø Ø Missing data frequencies Item Frequency Distributions – Floor/Ceiling effects ♦ Ø Eliminated item - “We had sex any time and any place” Ability to detect known group differences ♦ T-test - parametric ♦ Area Under an ROC Curve - nonparametric 13

Item Reduction and Domain Identification v Principle components analysis and Factor analysis: Ø Ø Unidimensional domains – measuring some facet of same underlying construct Items loading across multiple factors or small loadings (<0. 4) across all factors are eliminated v Multi-trait analyses – Item-total correlations to assess: Ø Ø Convergent/Divergent Validity – Items correlate more with their own domain than with other domains Software available – Multi-trait Analysis Program 14

Item Reduction - Factor Analysis Factor Item I wanted to avoid sex I avoided having sex I looked forward to sex Sex didn’t matter to me I didn’t care about sex at all Desire Responsiveness Disinterest 0. 34 0. 55 0. 33 0. 21 0. 79 0. 21 0. 58 0. 51 0. 15 0. 23 0. 72 0. 20 0. 24 0. 72 15

Evaluating an Instrument - Validity v Validity - Property of measuring what is intended to be measured Ø Ø Ø Content validity – adequate sampling and representation of relevant disease characteristics Concurrent validity – Good correlation with a “Gold Standard” Construct validity – Extent to which an instrument measures the underlying constructs purported to represent 16

Construct Validity v Convergent validity - Instrument correlates with other measures off related aspects v Divergent validity - Instrument is not correlated with measures on unrelated aspects v Known-groups validity Ø Ability to distinguish groups known to differ Ø Treatment Sensitivity 17

Treatment Sensitivity v Decreases in Distress significantly greater on Testosterone than on Placebo 18

Desire score differentiates normal libido women from low libido women 19

Evaluating an Instrument - Reliability v Reliability – agreement between two or more measures of the same thing Ø Ø Test-retest reliability – reproducibility of score over separate measurement occasions ♦ Pearson correlation coefficient ♦ Intraclass correlation coefficient Internal consistency reliability – homogeneity of items within a domain ♦ Cronbach's alpha 20

References v Fayers P. ; Hays R. Assessing Qo. L in Clinical Trials. Second edition: 2005. v Streiner, D. L. ; Norman, G. R. Health measurement scales: A practical guide to their development and use 2 nd Ed. Oxford University Press: 1995. v Shrout, P. E. ; Fleiss, J. L. Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 1979, 86, (2), 420 – 428. v Nunnally, J. C. ; Bernstein, I. H. Psychometric Theory, 3 rd ed. Mc. Graw. Hill: New York, 1994. v Juniper, E. F. ; Guyatt, G. H. ; Jaeshke, R. How to develop and validate a new health-related quality of life instrument. In Quality of life and pharmacoenconmics in clinical trials, 2 nd Ed. , Soilker, B. , Ed. ; Lippincott. Rqven: Philadelphia, 1996; 49 -56. v Rodenberg, C. A. ; Kuznicki J. ; Yiu G. Instrument Development and Validation. In Encyclopedia of Biopharmaceutical Statistics. 2002 21