Guidelines for Data Presentation Objective Provide a framework

Guidelines for Data Presentation

Objective • Provide a framework that can be utilized as a tool for the advancement of standardized data presentation

Data Guidelines Experimental Design Sample Procurement Sample preparation Fix/Perm Which Fluorophore Controls Isotype? Single color FMO Instrumentation Appropriate Lasers Appropriate Filters Instrument Settings Lin vs Log Time A, W, H Interpretation Mean, Median %+ CV SD Signal/Noise Gating Analysis Presentation Histogram Dot Plot Density Plot Overlay Bar Graph

First, Lets address the problem • Data analysis incorporates many disciplines including instrumentation, statistics, biology, and photonics. Often times knowledge in one of the above is missing • Many different instruments and software packages are available. • Historical precedento Unfortunately there is a large body of work published with poor data and no clear guidelines

Some Examples of Poor Data Presentation -Arbitrary and difficult to replicate gate -On axis data difficult to visualize, interpret, and review 2011 Nature article

Some Examples of Poor Data Presentation -e. Bioscience product literature -Normal Peripheral blood stained with listed reagents -That’s some bright CD 19 and dim CD 3 -Ratio between B and T seems off for normal blood

Some Examples of Poor Data Presentation From Nature Medicine, 1998. Human stem cells were injected into NOD/SCID mice and were reported to reconstitute multiple lineages. Myeloid B Cells T Cells… CD 4 & CD 8

Some Examples of Poor Data Presentation “Medium-to-high FS”? Did they backgate to ensure this was the correct gate?

Some Examples of Poor Data Presentation An isotype control for two channels? Which one? (CD 45 was on yet a third channel; no control for that? ) How was gate actually defined on this control? Impossible to estimate the amount of background staining in this histogram: need a gate to express it! Other graphs are shown as bivariate displays, causing difficulty to translate. % Pos?

Some Examples of Poor Data Presentation Why are cells expressing both markers? If these are myeloid origin, then why is a lymphocyte gate (“R 1”) applied? The cells on the diagonal look like nonspecific staining, and in fact were probably present in the isotype control.

Some Examples of Poor Data Presentation Nearly 100% of cells are expressing CD 19. If so, then there is no “room” left over for other lineages… The data appears self-contradictory. But without percentages, we cannot tell.

Some Examples of Poor Data Presentation Same problem as for “myeloid” cells: The CD 2+CD 3+ cells appear to be non-specifically-stained. The CD 4 and CD 8 distributions don’t look like typical mature T cells… and what about the CD 4+CD 8+.

Some Examples of Poor Data Presentation Why do graphs “e” and “h” have so many events compared to graphs “d”, “f”, and “g”? R 1 + R 2 (2. 5%) represents very few events…

Some Examples of Poor Data Presentation FITC and PE appear to be over-compensated.

An Example of Poor Data Presentation: Summary Example 1 Critical analysis of this figure shows that it does not support the contentions of the authors. This does not mean that the authors were wrong. Reviewers should have demanded a more rigorous example dataset… but perhaps the reviewers were not FACS experts. Guidelines can educate Unfortunately, this example is neither unique… nor even uncommon.

Research Misconduct Inquiry The Division of Investigative Oversight, Office of Research Integrity is currently swamped with request for flow cytometry related Research Misconduct Inquiries. Currently a majority of these cases display blatant intentional fraud. However there is a significant trend pushing for flow related guidelines, and the onus on investigators for proper representation of data is growing.

Research Misconduct is defined by law: 42 CFR Sections 50 and 93. Sections 93. 103 & 104: Research misconduct is defined as fabrication, falsification, or plagiarism … in reporting research results. Falsification is manipulating … changing or omitting data or results such that the research is not accurately represented in the research record. Misconduct can be committed intentionally, knowingly, or recklessly.

There is no wrong way to analyze your data Meaning- Investigators are free to choose: • Which plot types for display • Placement of gates for analysis • Which statistics • # events to display or collect • Which software package to use • How many times you reanalyze

There is definitely a wrong way to analyze your data Meaning- Investigators decisions can lead to incorrect data generation or interpretation: • Inappropriate gates for analysis (lymphocyte gate for CD 15 staining, or inconsistent gates) • Misleading or inconsistent plots for display • Inappropriate controls (e. g. using isotype for gating) • Inappropriate number of events collected (too few events for meaningful and accurate statistical comparison)

Implementation of Guidelines by J. Exp. Med. A set of guidelines for publication of flow cytometry data has been implemented by the Journal of Experimental Medicine All papers submitted for review will be required to comply with the guidelines, with submission of supplementary information, in order to be reviewed. Papers with sophisticated flow cytometric analysis may undergo an independent review to ensure the appropriateness of the analysis and presentation.

MIFLow. Cyt Minimum Information about a flow Cytometry Experiment ISAC Recommendation The fundamental tenet of scientific research is that the published results of any study have to be open to independent validation or refutation. The MIFLow. Cyt establishes criteria for recording and reporting information about the flow cytometry experiment overview, samples, instrumentation, and data analysis. It promotes consistent annotation of clinical, biological, and technical issues surrounding a flow cytometry experiment by specifying the requirements for data content and by providing a structured framework for capturing information

Guidelines: Why do we need them? • A consistent presentation style ensures better communication of data to readers and listener • Speaking a common language • Faster interpretation; understanding nuances • Provides a level of confidence that the data has been appropriately generated analyzed • Allows reviewers and readers to focus on the point of the presentation, avoiding distractions from inappropriate or inconsistent presentations

Guidelines: What they are NOT They will not define how to do science or how to analyze and interpret the data. In most cases, they are not requirements; they simply codify the “between the lines” information. They will not prevent nor reduce purposeful fraud. They can reduce reckless science. They can reduce confusion and ambiguity within published data

Principles and Guidelines A few examples of the principles and guidelines for data presentation follow. Introduction

Principles and Guidelines Hardware/Software Information about the instrument configuration should be provided Why: Different configurations (laser, filters, etc. ) can result in very different sensitivities, compensation requirements, etc. Some experiments (for example, fluorescence intensity comparisons across different days) require that the instrument be carefully calibrated. Interpretation of the significance of the results may require knowledge of these procedures.

Instrument • Manufacturer • Identify the FACS instrument and software used to collect, compensate and analyze the data. • Include Model and Version where more than one exists. • Light source • Type • Wavelength • Power • Optics- Band pass, Long Pass, 530/30

Instrument Configuration Hardware/Software Providing instrument configuration is a delicate balance between providing sufficient information as to be useful vs. providing too much that is not helpful. Instrument configuration can be summarized in three sections: • Optical • QA/QC • Compensation There is no “right” procedure (but there are “wrong” procedures for some kinds of experiments). Knowing instrument configuration is necessary to fully interpret data.

Instrument Configuration: Optical Hardware/Software The optical configuration determines what fluorescence measurements were made by the instrument. There are two tables: one for lasers, the other for detectors. FACS core facilities can create these tables and supply them to users

Instrument Configuration: QA/QC Hardware/Software Knowledge of the QA/QC procedures are necessary to understand how data analysis was performed. Do the gates move from experiment to experiment? Are MFI calculations compared between experiments? Is sensitivity equivalent across experiments? Relevant QA/QC procedures can likely be summarized by a limited set of options that authors select from: o No daily QC (i. e. , fire up the instrument and hope that yesterday's settings are close enough) o Alignment using beads: Set the instrument so that the same output fluorescence is observed on each channel every day o Set the instrument up to the same voltages and settings each day (record beads for QA) o Set the instrument up so that unstained cells are in the first decade of fluorescence

Hardware/Software Instrument Configuration: Compensation A very brief description of how compensation was accomplished is all that is needed. • What were the controls? (Beads, cells, combinations) • Was compensation manual or automatic? • What software was used to compensate? • Was manual adjustment of compensation necessary? This helps reviewers interpret distributions that they may think are improper compensation.

Principles and Guidelines Graphs-General Graph axis labels should include (at a minimum) the reagent being measured Why: Interpretation of the graph is much faster; the reader does not have to translate each label. In the case of fluorescent antibodies, both the specificity and the fluorochrome should be indicated. Do not use “FL 1” or “P 1” as a label.

Fluorescent Reagent Description • • • What is binding target Reporter (Fluorochrome) Clone name or number Reagent Manufacturer Reagent catalogue number

Principles and Guidelines Graphs-General The number of events displayed in any graph should be indicated Why: • The number of events making up a display can impact on the visualization of the display • The number of events should be considered when interpreting the precision of the analysis

Graphs-General Annotating Graphs Indicate with a simple number within or near each graphic, or list in the Figure Legend. Consistent use of color helps minimize extraneous text Axis labels show both the measurement and the fluorochrome Figure 001. 01

Scaling or Axis labels • Show all parts of the plot axis that indicate the scaling that was used, (Lin, Log, Bi-exponential) • Numerical values for axis “ticks” an be eliminated except when necessary to clarify the scaling.

Principles and Guidelines Graphs-General To convey quantitative representation of subsets from graphical displays, a calculated frequency of gated events must be displayed. The graph itself cannot convey such information. Why: Depending on how many events are displayed, the appearance of a subset may be quite different. The only way to assess the frequency with accuracy is to provide a numerical value. Histograms can provide notoriously misleading information about frequencies.

Graphs-General Graphs Cannot Convey Frequencies 250 100 200 80 Gate # Cells % of Max 150 100 60 40 50 20 0 50 100 For. Sc 150 200 Two datasets. What is the representation of “large” (high forward-scatter) cells? Does the “red” distribution have more? Figure 001. 04

Graphs-General Graphs Cannot Convey Frequencies Events: 4, 922 Blue Which distribution has more cells? Red Figure 001. 04

Intensity measurement Explicitly define the statistic applied (mean, median, Geo mean

Principles and Guidelines Graphs-General The choice of smoothing and specific display type is up to the author. Choose whichever graph and display options most readily convey the information needed to interpret the experiments, but be consistent across all graphs within an analysis Why: There is no single “best” way to display data. Each display type has advantages and disadvantages. However, using different displays in different graphs may mislead readers because of the nuances of emphasis by each graph type.

Principles and Guidelines Whenever gated analyses are performed, an illustration of the gating process should be shown. Why: The way in which cells are gated can dramatically impact the analysis and interpretation, particularly when rare populations are involved. Backgating demonstrates how each gate has impacted the analysis, and can demonstrate that the gating process has not artefactually selected for the subsets being analyzed. The gating “tree” teaches readers how to analyze data when they do similar experiments. Gating

Principles and Guidelines Unless otherwise explicitly stated, gating is assumed to have been performed subjectively Why: By convention. Gating

Principles and Guidelines The use of control samples to set gates should be shown; the algorithm to place gates should be explicitly defined if it was not subjective Why: In many cases, subjective placement of gates is a reasonable way to analyze the data; interpretation will not be affected by minor relocations of the gate. However, some types of analysis require rigorous placement of gates to provide the most significant data. If gate placement was algorithmic, then it must be described and shown. Gating

Gate Placement Algorithms Purely subjective Illustration is always useful. Unlikely to be acceptable for quantitative fluorescence measurements, identification of dimly-expressing subsets; discrimination between overlapping subsets. Based on control stains (unstained, FMO, etc. ) The control sample must be shown, along with a description of how it was used to place the gate. If the gates move for different types of samples (e. g. , treated vs. untreated), then at least one example of each should be given. Objective algorithm. Detail the algorithm (e. g. , “Top 2% of events”; “Autogate defined by software”). Gating

Experimental and Sample Information • How were cell suspensions prepared o o Specific proteases Filtration Lysing agents Fix/Perm reagents

Implementation of Guidelines by J. Exp. Med. In addition to ensuring that primary data presentation conforms with the guidelines, authors will also be expected to submit a single additional supplementary section devoted to the flow cytometry. This section will include: • Table of instrument information (template provided online) • Gating tree example(s) • Gating control(s) • Additional analyses pertinent to the interpretation of the flow cytometric data

References Prefetto et al 2006 JIM Keeney et al 1998 Cytometry 30(5), 1997 MIFLow. Cyt 1. 0 http: //ucflow. blogspot. com/2011/04/display-transformation -and-flowjo. html (bi-exponential display) Cytometry A 783 A: 384 -385 Seventeen-colour flow cytometry: unravelling the immune system Stephen P. Perfetto, Pratip K. Chattopadhyay & Mario Roederer Nature Reviews Immunology 4, 648 -655 (August 2004)