How to Read Research Papers Xiao Qin Department

How to Read Research Papers? Xiao Qin Department of Computer Science and Software Engineering Auburn University http: //www. eng. auburn. edu/~xqin@auburn. edu Some slides are adapted from notes by Nitin H. Vaidya at the University of Illinois at Urbana-Champaign

Why read papers? • To know what’s happening • Avoid reinventing the wheel – does happen commonly, too many wheels already • Find interesting research topics

Why not to read papers? • Cannot read everything • Should not read everything • Can suppress innovation – once you see solutions using a particular theme, often hard to think differently

Read or not to read, that is the question • Read, of course • Know what’s important • Know what can be ignored without significant loss of information

Where to search? • When I was a Master’s student… • When I was a Ph. D. student… • Now…

Where to search papers? • IEEEXplore • ACM Digital Library

Where to search papers? • Google Scholar • Personal Web Pages

What to read? • Major conferences – Journals are a few years behind, but still can be useful • Tech reports from active research groups – need to know which groups to look up • Survey / overview papers – ACM Computing Surveys – CACM, IEEE Computer, Spectrum – more technical - IEEE Personal Communications, … – newsletters - ACM SIGCOMM, ACM SIGMOBILE, . . .

What’s in a paper [see also how to write papers: http: //www. auburn. edu/~xzq 0001/Write. Papers-Part 1. ppt] • • • Abstract Introduction Motivation Problem description Solution. . . Performance Analysis Conclusions Future Work

How to read a paper? Know why you want to read the paper • To know what’s going on (e. g. , scanning proceedings) – title, authors, abstract • Papers in your broad research area – introduction, motivation, solution description, summary, conclusions – sometimes reading more details useful, but not always • Papers you may want to improve on – read entire paper carefully

What to note? • Authors and research group – Need to know where to look for a paper on particular topic • Theme of the solution – Should be able to go back to the paper if you need more info • Approach to performance evaluation • Note any shortcomings

So this paper is in print. . . • Be skeptical • If it sounds too good to be true, it often is ……

More Advice …

Three phases to reading • Determine if there is anything interesting at all in the paper. • Determine which portion of the paper contains the interesting stuff. • Should I read the whole paper? – Read the whole paper if necessary.

Is there anything interesting? • Ideally, the abstract should tell you this, but frequently it does not. • Need to jump about – Read conclusion – Read introduction – Look at the bibliography – Glance at the TOC (if any)

Which portion contains interesting stuff? • A paper typically outlines its organization at the end of the introduction. – Use this to determine which portion contains the exciting stuff. • Quickly scan the titles of each sections in the paper.

Read the whole paper • Read with the following questions in mind – How can I use this stuff? – Does this really do what the author claims to do? – What if the assumptions and choices that the author made are discarded (or made invalid)? – More questions ……

Context and problem statement • What problems are the author trying to solve or trying to convince you of? • Are they important problems? – Why not? • What is the author’s thesis?

Related work evaluation • Does the author describe other work in the field? • If so, how does this research differ from the other work?

New idea • What new idea is the author proposing? – Architecture – Algorithm – Mechanism – Methodology – Perspective • Is the idea useful and practical?

What to evaluate? • What need to be evaluated to confirm the worthiness of the new idea? – Runtime – Throughput – Resource utilization – Model validation

How to evaluate? • How did the author go about conducting the evaluation? – Formalize and prove theorems – Run simulations – Artifact design and construction – Collect traces from existing systems

Was the evaluation correct and adequate? • How was the data collection done? • Do you agree with the analysis of data? • Do you agree with the conclusions about the data? • Do you have any new interpretation of the data? • Can you suggest new ways to evaluate the data?

Assumptions, drawbacks and extensions • Can you think of other aspects of the idea that need to be evaluated? • Can you think of extensions or modifications to the idea to improve it? • How would you evaluate your improvement?

Assumptions, drawbacks and extensions • Can you apply the idea or method of evaluation to your own project? • Do the authors make any assumptions that are not valid or realistic? • Can you come up with a more general solution that does not rely on one or more of the assumptions?

Future work • Does the author indicate how the work should be followed up on? • Does the paper generate new ideas? • Does the paper implicitly or explicitly provide a new way of doing other things or of thinking about problems?

Summary If you remember nothing else: • Where to search papers • Scan papers • Read papers with questions in mind • Download the slides from http: //www. auburn. edu/~xzq 0001/How to Read Papers - Qin. ppt

http: //www. auburn. edu/~xzq 0001

Questions