Statistical presentation in international scientific publications 3 Statistics

Statistical presentation in international scientific publications 3. Statistics by section Malcolm Campbell Lecturer in Statistics, School of Nursing, Midwifery & Social Work, The University of Manchester Statistical Editor, Health & Social Care in the Community 26 March 2008 Statistical presentation - 3. Statistics by section

3. Statistics by section Contents • 3. 1 The Abstract • 3. 2 The Methods • 3. 3 The Results • 3. 4 The Discussion • Follow the guidelines for studies – CONSORT for randomised controlled trials – TREND for non-randomised trials – STROBE for observational studies 26 March 2008 Statistical presentation - 3. Statistics by section 2

3. 1 The Abstract Statistical reporting starts here (some readers stop here!) • Quantitative Abstracts need statistical content – aims and type of study design – the location, setting and dates of data collection – the selection and number of participants – descriptions of interventions, instruments & outcomes – a summary of main findings (with p-values) – conclusions and implications • Should it be structured or unstructured? – please check the requirements of the planned journal! – plan a structured one & remove structure if necessary 26 March 2008 Statistical presentation - 3. Statistics by section 3

3. 2 The Methods The sections of Peat et al (2002, p 54 -63) • Questionnaires • Ethical approval – and informed consent • Study design – each has own strengths and limitations, and determines the statistical approach • Participants – details, and how developed, validated, tested & administered • Interventions – details and how administered – randomisation, allocation concealment & blinding • Clinical assessments – how clinical info collected – recruitment, sampling frame, inclusion/exclusion • Statistical methods criteria, logic behind – how the data were analysed groupings (assumptions, tests), and with • Sample size what software – bivariate analyses • [+ Main outcomes? ] 26 March 2008 Statistical presentation - 3. Statistics by section 4

The Methods (part 1 of 3) How was the study carried out? (CONSORT/TREND/STROBE) • Describe the methods used, including (where appropriate) – the specific aims of the study (hypotheses tested) – the type of design used (in common terminology) – clear descriptions of any interventions, main and any secondary outcome measures – source of participants with eligibility criteria • description of target population and study population – how access to the participants was arranged – methods used for randomisation, allocation concealment or probability sampling 26 March 2008 Statistical presentation - 3. Statistics by section 5

The Ugly (Methods) No inferences please, we’re purposive • Randle (2003) – Changes in self-esteem during a 3 -year pre-registration Diploma in Higher Education (Nursing) programme, J Clinical Nursing 12, 142 -143 • “Purposive samples were recruited from all four branches of the nursing programme…” • “Analyses were performed using SPSS (SPSS Inc. , Chicago, IL, USA) to see if there were significant differences or correlations between respondents’ scores and variables: cohort, age of student, nursing branch studied and referrals at any of the academic summative assessment stages. ” • no numerical results reported at all! – no counts, no percentages, no descriptive statistics, no test results… 26 March 2008 Statistical presentation - 3. Statistics by section 6

More Methods (part 2 of 3) How was the study carried out? • … and report (where appropriate) … – details of ethical approval and participants’ consent – dates, settings and locations for data collection – sample size calculations (see your research proposal? ) • whether the study was powered to detect a clinically important result • not necessary for pilot studies, time-restricted studies or audit-based analyses • can be difficult to do if there is no prior information on which to base calculations • many papers do not justify their sample size (eg Altman, 1998) – samples usually small, occasionally large 26 March 2008 Statistical presentation - 3. Statistics by section 7

The Ugly (Methods) No sample size calculations • Gillespie and Melby (2003) – Burnout among nursing staff in accident and emergency and acute medicine: a comparative study, J Clinical Nursing 12(6), 842 -851 • 28 questionnaires sent to nurses in A&E and 28 to nurses in acute medicine in an NHS Trust – no details of where or when, no justification for 28 – 20 replies from A&E, 16 from acute medicine • “The sample was small in numbers and findings cannot therefore be generalised beyond this sample of participants. ” • [also poor presentation of results] 26 March 2008 Statistical presentation - 3. Statistics by section 8

The Good Sample size calc. • Young et al (2005) – A prospective baseline study of frail older people before the introduction of an intermediate care service, HSCC 13(4), 307 -312 26 March 2008 Statistical presentation - 3. Statistics by section 9

The (slightly) Bad (Methods) Kim and Ooooh-no - we forgot about the dropout… • Kim and Oh (2003) – Adherence to diabetes control recommendations: impact of nurse telephone calls, J Advanced Nursing 44(3), 256 -261 • “For repeated measures ANOVA, for an effect size of 0. 60, at a power of 0. 80, and an alpha level of 0. 05, 25 subjects in each group were needed to ensure an adequate trial for 1% reduction of Hb. A 1 c levels…” • “A total of 50 patients… agreed to participate. They were randomised by a toss of a coin… Only 36 subjects completed the entire study… Four moved to another city, 10 refused before completing the post-test. ” • [otherwise nicely reported (intervention group did show a drop in mean Hb. A 1 c level from 8. 8% to 7. 6%)] 26 March 2008 Statistical presentation - 3. Statistics by section 10

Even more Methods (part 3 of 3) How was the study carried out? • … and finally (usually in a paragraph at end of the section) report – statistical methods used for data analysis • with references for those that are not well-known – the name and version of software used for data management and analysis • not in CONSORT or STROBE statements but is in TREND statement • make sure the software is referenced appropriately according to the planned journal – some require formal references, some don’t 26 March 2008 Statistical presentation - 3. Statistics by section 11

The Ugly (Methods) Wrong statistical tests probably used! • Paxton et al (1996) – Evaluating the workload of practice nurses: a study, Nursing Standard 10(21), 33 -38 • study comparing workload of same 34 practice-employed and health board attached nurses before and after introduction of the New General Practitioner Contract • “Data were coded analysed using the Statistical Package for the Social Sciences (SPSS) and significance between categorical variables determined by the chi square statistic. ” Statistical methods used for other variables (% of time, hours per FTE) not described • [also no sample size calculations] • no test statistics reported; only p-value ranges (see later) • testing should have taken the paired nature of the data into consideration 26 March 2008 Statistical presentation - 3. Statistics by section 12

The Good Statistical methods • Christensen et al (2005) – Recruitment of religious organisations into a community-based health promotion programme, HSCC 13(4), 313 -322 26 March 2008 Statistical presentation - 3. Statistics by section 13

The (ever-so-slightly) Bad (Methods) SPSS is a four-letter word, not an acronym • Chan and Yu (2004) – Quality of life of clients with schizophrenia, J Advanced Nursing 45(1), 72 -83 • “Data were entered into the Statistical Package for Social Science (SPSS) version 10. 0. ” • SPSS stopped being “the Statistical Package for the Social Sciences” certainly by the mid-1980 s when the MSDOS version, SPSS/PC+ was introduced • it should be referenced as “SPSS™” for first use and “SPSS” subsequently, cited in a form such as SPSS (2003), SPSS for Windows, Rel. 11. 5. SPSS Inc. , Chicago IL see http: //www. spss. com/corpinfo/faqs. htm • [otherwise statistics are well reported!] 26 March 2008 Statistical presentation - 3. Statistics by section 14

3. 3 The Results The template of Peat et al (2002, p 65) • Paragraph 1 • Paragraphs 3 to n-1 – describe study sample – bivariate analyses – who did you study? – what is the relation between the outcome and explanatory variables [taken one at a time]? • Paragraph 2 – univariate analyses – how many participants had what? • Last paragraph/s – multivariate analyses – what is the result when the confounders and effect modifiers have been taken into account? 26 March 2008 Statistical presentation - 3. Statistics by section 15

Detailed Results What was found? – text (eg Altman et al, 2000) • A description of the statistical findings in simple language, including (where appropriate) – numbers of participants and participation rates – characteristics of/baseline information on participants • including baseline comparison of any groups – known information on non-participants – the results of preliminary analyses, in lesser detail – results of main analyses, as determined by the aims of the study, in full detail – results of any secondary analyses, in lesser detail – should be clear, factual and concise (Kelley et al, 2003) • The most exciting part of the paper! 26 March 2008 Statistical presentation - 3. Statistics by section 16

The Good (Results) Simple recruitment flow chart • CONSORT/TREND/STROBE requirement • Peters et al (2004) – Factors associated with variations in older people’s use of community-based continence services, HSCC 12(1), 53 -62 26 March 2008 Statistical presentation - 3. Statistics by section 17

The Good (Results) Another flow chart • Perry and Mc. Laren (2004) – An exploration of nutrition and eating disabilities in relation to quality of life at 6 months poststroke, HSCC 12(4), 288 -297 26 March 2008 Statistical presentation - 3. Statistics by section 18

Reporting numbers How do I report numerical information? • Unless readability is compromised, report numbers with percentages – eg 123 (45%) • Report estimate of centre with estimate of spread – eg means with SDs, medians with ranges or IQRs • Report test results in full with supporting statistics so reader can understand the findings – test results include label for test statistic to indicate test, value of test statistic (eg t =, M-W Z =, 2 =), degrees of freedom and actual p-value (all where applicable) – report even results non-significant, assuming the test was important (if not important, why do it? ) 26 March 2008 Statistical presentation - 3. Statistics by section 19

Reporting numbers continued Golden rules for reporting numbers – Peat & Barton (2005) • In text, give numbers • Rules for sample size & %: – with units (eg cm) as numbers – < 20: use numbers not %s – < 10 as words – < 100: % to nearest whole number – ≥ 10 as numbers – at start of sentence as words – > 100: % to 1 decimal place • Use a 0 before decimal point for • Use one more decimal place than unit of measurement numbers < 1 when reporting descriptive • No space between number and statistics % sign but space between • Report last decimal place if 0* number and unit • Use 2 decimal places for most • Report p-values to 3 decimal places or 2 significant figures, test statistics & correlations* or p < 0. 001 if very small* * My rules! 26 March 2008 Statistical presentation - 3. Statistics by section 20

The Good Results in the text • Young et al (2005) – A prospective baseline study of frail older people before the introduction of an intermediate care service, HSCC 13(4), 307 -312 26 March 2008 Statistical presentation - 3. Statistics by section 21

The Good More results in text • Trappes-Lomax et al (2006) – Buying Time I: a prospective, controlled trial of a joint health/social care residential rehabilitation unit for older people on discharge from hospital, HSCC 14(1), 4962 26 March 2008 Statistical presentation - 3. Statistics by section 22

More Results What was found? - tables and figures • Summarise findings in tables and figures in a clear, consistent layout, including (where applicable) – flow chart of recruitment (CONSORT/TREND/STROBE) – characteristics of the participants (eg by group) – results of main analyses (maybe secondary analyses) • Each table or figure should have a specific role • Arrange results clearly and appropriately – eg tables with groups by column, variables by row for reading left to right – number of cases should be shown 26 March 2008 Statistical presentation - 3. Statistics by section 23

And more Results More about tables and figures • Tables and figures should be self-contained – imagine that a reader may want to use one in a talk – titles should be self-explanatory – include an interpretation in the text to guide the reader through the table • Can a figure be summarised in a table or a sentence? – tables are more compact than figures – sentences are more compact than tables • Avoid charts with unnecessary third dimension – perspective can distort interpretation 26 March 2008 Statistical presentation - 3. Statistics by section 24

The Good A good summary table • Young et al (2005) – A prospective baseline study of frail older people before the introduction of an intermediate care service, HSCC 13(4), 307 -312 26 March 2008 Statistical presentation - 3. Statistics by section 25

The Good (Results) Another good summary table • Armstrong and Earnshaw (2005) – A comparison of GPs and nurses in their approach to psychological disturbance in primary care consultations, HSCC 13(2), 108 -111 26 March 2008 Statistical presentation - 3. Statistics by section 26

The Bad (Results) Tables that require an explanation 1 (see next slide) • Söderhamn et al (2001) – Attitudes towards older people among nursing students and registered nurses in Sweden, Nurse Education Today 21, 225 -229 • [also no sample size calculation – 86 first year & 65 second year students, 41 registered nurses – insufficient details of sampling: authors call it a convenience sample with a response rate of 100%, but it may have been interpretable as a representative random sample for inferences] • tables give p-value ranges but no test statistics, & include abbreviations that need the reader to refer to the text – you might make an educated guess that they were subscales of the scale mentioned in the table title 26 March 2008 Statistical presentation - 3. Statistics by section 27

The Bad (Results) Tables that require an explanation 2 26 March 2008 Statistical presentation - 3. Statistics by section 28

The Good (Results) A table with descriptive statistics and test results • Evans et al (2005) – The impact of ‘statutory duties’ on mental health social workers in the UK, HSCC 13(2), 145 -154 26 March 2008 Statistical presentation - 3. Statistics by section 29

The Bad (Results) Could have done it better with tables • Chang et al (2002) – A continuing educational initiative to develop nurses’ mental health knowledge and skills in rural and remote areas, Nurse Education Today 22, 542 -551 • [also does not follow IMRa. D structure – vague characteristics of participants in Methods section; response rate is given in Methods and Results (twice) • no sample size calculation (202 questionnaires returned from 303 sent)] • nurses’ responses to 7 -point Likert statements on mental health program reported repeatedly in text as % “A/SA” (agree or strongly agree): could have been presented more compactly with more detail in one or more tables 26 March 2008 Statistical presentation - 3. Statistics by section 30

The Good A double bar chart • Klinkenberg et al (2005) – The last three months of life: care, transitions and the place of death of older people, HSCC 13(5), 420 -430 26 March 2008 Statistical presentation - 3. Statistics by section 31

The Ugly (Results) Read off the scale – emotional exhaustion 1 (see next slide) • Gillespie and Melby (2003) [again] – Burnout among nursing staff in accident and emergency and acute medicine: a comparative study, J Clinical Nursing 12(6), 842 -851 • [also small sample size, not justified (see earlier) • software used for analysis not named (probably Excel)] • clustered bar charts used to present frequency counts instead of tables – have to read number in each bar from vertical scale • accompanying text quotes percentages only, while bar charts have counts axis • [also actual p-values but no test statistics; Kruskal-Wallis results not explained] 26 March 2008 Statistical presentation - 3. Statistics by section 32

The Ugly (Results) Read the numbers off the scale – emotional exhaustion 2 26 March 2008 Statistical presentation - 3. Statistics by section 33

The Bad (Results) Chart that requires an apology 1 (see next slide) • Salanterä and Lauri (2000) – Nursing students’ knowledge of and views about children in pain, Nurse Education Today 20, 537 -547 • [also characteristics of the sample given in Methods • method used to divide students into two groups (those who thought their knowledge was good and those who thought it was poor) not clearly defined • comparison of groups with no supporting statistics and test results concentrating on significant findings only] • important results displayed as 3 D bar charts, where percentages have to be read against axis – would have been better presented as tables 26 March 2008 Statistical presentation - 3. Statistics by section 34

The Bad (Results) Chart that requires an apology 2 26 March 2008 Statistical presentation - 3. Statistics by section 35

The Good A good line plot • Roelands et al (2005) – Knowing the diagnosis and counselling the relatives of a person with dementia: the perspective of home nurses and home care workers in Belgium, HSCC 13(2), 112 -124 26 March 2008 Statistical presentation - 3. Statistics by section 36

The Good Error bars (a confidence interval plot) • Armstrong and Earnshaw (2005) – A comparison of GPs and nurses in their approach to psychological disturbance in primary care consultations, HSCC 13(2), 108111 – better if superimposed on bars 26 March 2008 Statistical presentation - 3. Statistics by section 37

The Good A clever box plot • Pollard et al (2004) – Collaborative learning for collaborative working? Initial findings from a longitudinal study of health and social care students, HSCC 12(4), 346 -358 26 March 2008 Statistical presentation - 3. Statistics by section 38

3. 4 Discussion The template of Peat et al (2002, p 87) • Paragraph 1 • Paragraphs 3 to n-1 – what did this study show? – address the aims shown in the Introduction[/Methods] • Paragraph 2 – strengths and weaknesses of methods – discuss how the results support the current or refute current knowledge literature • Final paragraph – future directions – “so what? ”, “where next? ” – impact on current thinking or practice 26 March 2008 Statistical presentation - 3. Statistics by section 39

The Discussion 2 Statistics up for discussion – Kelley et al (2003) • Interpret and discuss findings – compare findings with those of other studies – critical reflection on both results and data collection – assess how well the study met the research question – describe problems encountered – honestly judge the limitations of the work • the authors are the ones best placed to guide the reader • Present conclusions and recommendations 26 March 2008 Statistical presentation - 3. Statistics by section 40