Communicating Research Results Research Methods for Public Administrators

Communicating Research Results Research Methods for Public Administrators Dr. Gail Johnson Dr. G. Johnson, www. Research. Demystified. org 1

Transition: Back To The Beginning n Research planning is the funnel into specifics n Data collection and number crunching mean working with details n Making sense of the data means returning to the beginning—to answer the questions that prompted the study Dr. G. Johnson, www. Research. Demystified. org 2

Transition: Back To The Beginning n In this phase the key questions for the researchers: n What are the answers to the research questions? n What are the major messages based on the research results I want to convey? n Who is the primary audience? n How can I package my research results so it meets the requirements of my audience? Dr. G. Johnson, www. Research. Demystified. org 3

Transition: Back To The Beginning n “What’s my point? ” is the mantra n Whatever is not on point gets cut n I keep a “parking lot” file for those great findings, brilliant phrases, and blinding insights that do not contribute to my point n This feels better than hitting the delete key Dr. G. Johnson, www. Research. Demystified. org 4

Transition: Back To The Beginning n Conversations with the stakeholders or the boss are useful in clarifying their primary interests and concerns n It is often useful to share preliminary findings as part of those conversations n Especially if it the data are not what was expected or if the data are more negative than anticipated Dr. G. Johnson, www. Research. Demystified. org 5

Consider: The Use of Research Results n Is it intended to n Inform interested parties? n Prompt action? n More research: suggestions based on what you learned and the new questions that need to be addressed n New ideas for methodology n Program/policy changes: – Suggestions based on your study – Recommendations Dr. G. Johnson, www. Research. Demystified. org 6

Consider Dissemination Strategy n How will you get the information out? n Depends on situation n Do you intend people to use your research results? n Written Report n Presentation with sponsor/client n Presentation with people in the community n Place, time, publicity n Press release n Web presence Dr. G. Johnson, www. Research. Demystified. org 7

Communication Basics n The goal is to communicate-- not to impress n Make it easy for the reader to get your point n Keep your purpose and audience in mind n Keep it simple and free from jargon n Consider your presentation from your audience’s point of view Dr. G. Johnson, www. Research. Demystified. org 8

Communication Basics n Synthesis: organize a report around major themes or research questions Not a chronology of who was interviewed n Not a presentation of all the survey results n Not a listing of what each book or article said n n The researcher is now telling the story of the data n The analysis should reveal specific themes n The researchers decide on the major messages they want their audience to understand Dr. G. Johnson, www. Research. Demystified. org 9

Communication Basics n Provide enough information about the research methods so others can judge its credibility n Include all information about methods, data collection, sampling, etc n Briefly in body of the report, more detail in an appendix Dr. G. Johnson, www. Research. Demystified. org 10

Communication Basics n Always state the limitations of the study n Every study has limitations n Explain trade-offs necessary to conduct the study under the particular constraints in the field, the things that could not have been anticipated or controlled n Do it before your critic challenges you on the limitations n Conclusions and recommendations should be mindful of those limitations Dr. G. Johnson, www. Research. Demystified. org 11

Presenting Results: The Options n Writing Reports for the Public n Executive Summary n Charts and Tables n Presentations Dr. G. Johnson, www. Research. Demystified. org 12

Discussion n Think of the best report that your have read. n What made it the best? Dr. G. Johnson, www. Research. Demystified. org 13

Overview: Written Reports n This is a story n Set up the tension: research questions, issues, and concerns n Provide the clues: findings section n Tell them who did it: conclusion Dr. G. Johnson, www. Research. Demystified. org 14

Overview: Written Reports n Engage reader from the start: what’s the hook? n Organize around research questions or themes n Place major points up front n Report data findings: simple, easy to follow Dr. G. Johnson, www. Research. Demystified. org 15

Overview: Written Reports n Provide enough information about your research methods so others can judge its credibility n Place technical information in an appendix n State limitations of study n Never make recommendations without evidence n Who specifically should do precisely what? Dr. G. Johnson, www. Research. Demystified. org 16

Overview: Written Reports n Cut draft by 10 percent to sharpen the focus n Leave time to revise, revise n Have a cold reader and a knowledgeable reader review the draft n Consider their comments and revise again Dr. G. Johnson, www. Research. Demystified. org 17

The Generic Report Structure n Introduction n Purpose n Issue, concerns n Why is this important n Background n Context n What is Known Dr. G. Johnson, www. Research. Demystified. org 18

The Generic Report Structure n Methodology n Brief in body of report n Who, how many n Details can go in appendix n Findings n Present data so audience can understand n Present data selectively: what’s your point? Dr. G. Johnson, www. Research. Demystified. org 19

The Generic Report Structure n Conclusions n Tie back to your research questions. n Recommendations, if any. n Make sure you have presented the evidence to support the recommendations. Dr. G. Johnson, www. Research. Demystified. org 20

Executive Summary n Essential for busy readers n Short: 1 -4 pages n Writing short is harder than writing long! n Bottom line focused: major findings and recommendations n Present in bullet format n Refer them to report or appendix for more detail n Ruthless adherence to “What’s my point? ” Dr. G. Johnson, www. Research. Demystified. org 21

Executive Summary Format n Executive Summary follows report structure: n Brief: n n n overview: background and purpose of the study n the “hook” Description of major questions, issues Description of research methods Major Findings Major Conclusions Major Recommendations Dr. G. Johnson, www. Research. Demystified. org 22

Conveying Numbers Through Charts and Tables One picture is worth a thousand words. Dr. G. Johnson, www. Research. Demystified. org 23

Charts and Tables n Purpose to: n Describe n Explore n Tabulate n Compare n Present data simply and accurately n Make data coherent and to the point Dr. G. Johnson, www. Research. Demystified. org 24

Charts and Tables n Benefits: n High Impact n Audience acceptance of data n Increased memory retention n Highlights the message n Visually interesting—breaks up text Dr. G. Johnson, www. Research. Demystified. org 25

Charts and Tables n Charts: Better for presenting the message. n Tables: Better for presenting data Dr. G. Johnson, www. Research. Demystified. org 26

Chart Terminology Chart Title: What’s the Point? Y Axis label --Percent Key: X Axis Label—Time Periods Source and Date Dr. G. Johnson, www. Research. Demystified. org 27

Chart Options Line: trends over time Dr. G. Johnson, www. Research. Demystified. org 28

Chart Options Bar chart: percent distribution Dr. G. Johnson, www. Research. Demystified. org 29

Chart Options Cluster bar chart: comparing several items Dr. G. Johnson, www. Research. Demystified. org 30

Chart Options Combination: lines and bars Dr. G. Johnson, www. Research. Demystified. org 31

Chart Options Pie Chart: parts of a whole, slices always add to 100% Dr. G. Johnson, www. Research. Demystified. org 32

Effective Charts n Easy to read n Use upper and lower cases (not ANGRY caps) n Only use a few type faces n Appropriate for the delivery n Avoid busy patterns n Use white space n Keep the chart simple Dr. G. Johnson, www. Research. Demystified. org 33

Effective Charts n Keep scales honest n Use title to convey message n Provide sufficient data with the chart so the message is there n Identify source of data n Put supporting data in an appendix Dr. G. Johnson, www. Research. Demystified. org 34

Judicious Use of Charts: Too Much? Dr. G. Johnson, www. Research. Demystified. org 35

Effective Tables n Simple and accurate n Clearly label rows and columns n No abbreviations n Show percents n Round to nearest whole numbers when showing percentages and $money n Show total numbers (basis of the percents) n Identify the source of the data Dr. G. Johnson, www. Research. Demystified. org 36

Table Title With A Message Row Title Column Title (N= 300) 50% Column Title (N=360) 40% Row Title 55% 60% Total 100% Dr. G. Johnson, www. Research. Demystified. org 37

Discussion n Think of the best presentation you have ever heard. n What was that presentation about? n What made it the best? Dr. G. Johnson, www. Research. Demystified. org 38

Oral Presentations n Prepare: n Who is your audience? n What do you want them to remember? n How much time will you have? n What’s the delivery resources available? n What few handouts, if any? n Rehearse, time, get feedback, Dr. G. Johnson, www. Research. Demystified. org 39

Oral Presentations n Organize: n Tell them what you will tell them n Tell them what you told them n Avoid complex language, detailed data n If possible, keep interactive n Be prepared for questions Dr. G. Johnson, www. Research. Demystified. org 40

Avoid Death By Powerpoint n Use only as many slides as is necessary—key them to major points n Simple and clear, big font, few points per slide n Keep bells and whistles to a minimum: too distracting n Don’t read from powerpoints Use as a tool to prepare the presentation n Use as an outline when giving the presentation n Dr. G. Johnson, www. Research. Demystified. org 41

Other Presentation Options n Poster Board n Useful when you are at a conference or group event n It is like a visual executive summary n Want to attract people, needs to be simple but clearly present the message Dr. G. Johnson, www. Research. Demystified. org 42

Other Presentation Options n Media: press releases n This are short, one page n “Think Smart just released a community assessment that found…. . ” n “For more info, contact…. . ” n Web presence n Websites n Blogs Dr. G. Johnson, www. Research. Demystified. org 43

Final Words About Recommendations n Only when you feel your research is solid and the evidence is strong n All research is flawed and there always limitations n “Strong” is a often judgment call n Those who like the results will be more willing to not ask tough questions n Those who do not like the results are likely to impugn the data and the methods, and maybe even question the honesty of the researcher Dr. G. Johnson, www. Research. Demystified. org 44

Final Words About Recommendations n Weasel words are useful if the study results are not definitive: n It appears that… n The results suggest that… n Policy makers might want to consider… n What else? Dr. G. Johnson, www. Research. Demystified. org 45

Final Words About Recommendations n Recommendations: specific and precise n Specifically who should do exactly what n “There ought to be a law” type of general prescriptions are often not useful n Should be tied to the research rather than opinion n But if it is opinion, it should be stated as such n Are the recommendation doable? Practical? Costly? n Be aware of the counter arguments and take into account when framing the recommendations Dr. G. Johnson, www. Research. Demystified. org 46

Final Words About Writing n Everyone has their own process n Some like outlines n Some like to organize their thoughts using 3 x 5 cards n Some like to do mindmaps n Some like to write their first draft in a stream of consciousness n Find what works best for you Dr. G. Johnson, www. Research. Demystified. org 47

Final Words About Writing n Just do it! n Set up a specific time and write n It does not have to be good n It just has to get out of one’s head n Then it can be changed and polished n It is not possible to write well if you have an active critic in your head—so turn it off. n The critic will get its turn once there is a solid draft Dr. G. Johnson, www. Research. Demystified. org 48

Final Words: Ownership n If you work in an organization, you do not own the material n Others will want to edit and make changes n One challenge is to discern the defensiveness that is related to the integrity of the work and the defensiveness that is related to one’s ego n There are many ways the same information can be conveyed—it is a style preference n It is not personal Dr. G. Johnson, www. Research. Demystified. org 49

Final Words: Ownership n The second challenge is overcoming the fear that protecting the integrity of the work will be career limiting It may be that your boss does not understand that criticism that those changes will create n Your job is to explain the likely scenario and suggest some options that may meet boss’s concerns without damaging the integrity n Sometimes changing a few words can dramatically alter the tone and meaning but still be an honest presentation of the data—and that will solve the problem n Dr. G. Johnson, www. Research. Demystified. org 50

Final Words: Presenting Unwelcome Information n It would be nice if researchers always discovered programs that work, highly motivated employees, and satisfied citizens n But it does not work that way n When the message, after careful analysis and review, is negative, the researchers should be prepared for attacks on the methodology. n Really, there was a case where someone attacked the 59% response rate as being too good as proof there was something wrong with the study results Dr. G. Johnson, www. Research. Demystified. org 51

Presenting Unwelcome Information n Sometimes it helps to meet with the stakeholders to share the results and get their perspectives n Sometimes they will bring forth new data that shed light on the situation n Sometimes they are aware of the situation and have some possible remedies to be considered Dr. G. Johnson, www. Research. Demystified. org 52

Presenting Unwelcome Information n Begin with the assets n It is easier to hear the concerns and issues that follow n Focus on major problems for which there might be remedies n Word choice matters n Saying a program failed and implying the managers are inept is not a winning strategy n Better to say “the program encountered some challenges that limited success” n “Things are working but improvements are needed” Dr. G. Johnson, www. Research. Demystified. org 53

Presenting Unwelcome Information n But this is a difficult situation n Finding the balance between protecting the integrity of the data and the truthfulness of the negative findings while being sensitive to the issues and concerns of the officials is not easy n Consulting other professional about how to handle this situation may be warranted Dr. G. Johnson, www. Research. Demystified. org 54

Takeaway Lessons n For many, telling the story is the payoff n Several challenges n Making the story simple and clear even if the research methods are complex n Finding the balance between too much and too little information n Ensuring that bias does not creep in at the end—in drawing conclusions or making recommendations n Conveying unwelcome information in a way that can be heard and used Dr. G. Johnson, www. Research. Demystified. org 55

Takeaway Lessons n The final product, be it a report or an oral presentation, should have enough information about the research methods for a sophisticated reader to decide its credibility. Dr. G. Johnson, www. Research. Demystified. org 56

Creative Commons n This powerpoint is meant to be used and shared with attribution n Please provide feedback n If you make changes, please share freely and send me a copy of changes: n Johnsong 62@gmail. com n Visit www. creativecommons. org for more information Dr. G. Johnson, www. Research. Demystified. org 57