How to Read Research Papers Xiao Qin Department

How to Read Research Papers? Xiao Qin Department of Computer Science and Software Engineering Auburn University http: //www. eng. auburn. edu/~xqin@auburn. edu Some slides are adapted from notes by Nitin H. Vaidya at the University of Illinois at 1 Urbana-Champaign

Three Steps in a Research Project Identify a Problem Solve the Problem: a solution Present the solution Slide 2 of 38

Why read papers? • To know what’s happening • Avoid reinventing the wheel – does happen commonly, too many wheels already • Find interesting research topics • Write a good “related work” section Slide 3 of 38

Why not to read papers? • Cannot read everything • Should not read everything • Can suppress innovation – once you see solutions using a particular theme, often hard to think differently Slide 4 of 38

Read or not to read, that is the question • Read, of course • Know what’s important • Know what can be ignored without significant loss of information Slide 5 of 38

Where to search? • When I was a Master’s student… • When I was a doctoral student… • Now… ‹#› Slide 6 of 38

Where to search papers? (1) • IEEEXplore • ACM Digital Library ‹#› Slide 7 of 38

Where to search papers? (2) • Google Scholar • Personal Web Pages ‹#› Slide 8 of 38

Where to search papers? (3) • Top Conference Proceedings AREA: System Technology Premium: SOSP: ACM SIGOPS Symp on OS Principles OSDI: Usenix Symp. OS Design and Implementation Leading: ICPP: Intl Conf on Parallel Processing ICDCS: IEEE Intl Conf. Distr. Comp Systems Reputable: GLOBECOM: Global Comm ICCC: Intl Conf on Comp Communication Slide 9 of 38

Where to search papers? (4) • Top Journals AREA: System Technology Premium: Jnl of Parallel and Distributed Computing IEEE Trans on Parallel and Distributed Systems Distributed Computing Leading: Jnl of Network and Computer Applications Parallel Computing Reputable: ACM Mobile Computing and Communicatons Review Intl Jnl of High Speed Computing Slide 10 of 38

What to read? • Major conferences – Journals are a few years behind, but still can be useful • Tech reports from active research groups – need to know which groups to look up • Survey / overview papers – ACM Computing Surveys – CACM, IEEE Computer, Spectrum – more technical - IEEE Personal Communications, … – newsletters - ACM SIGCOMM, ACM SIGMOBILE, . . . Slide 11 of 38

Manage References Slide 12 of 38

References Management Software Biblio. Express Slide 13 of 38

References Management Software • • End. Note Web Zotero Ref. Works Reference Manager Mendeley See Comparison at: http: //workspace. imperial. ac. uk/library/Public/Reference_m anagement_software_comparison. pdf Slide 14 of 38

What’s in a paper [see also how to write papers: http: //www. auburn. edu/~xzq 0001/Write. Papers-Part 1. ppt] • • • Abstract Introduction Motivation Problem description Solution. . . Performance Analysis Conclusions Future Work Slide 15 of 38

How to read a paper? Three levels Know why you want to read the paper • To know what’s going on (e. g. , scanning proceedings) – title, authors, abstract • Papers in your broad research area – introduction, motivation, solution description, summary, conclusions – sometimes reading more details useful, but not always • Papers you may want to improve on – read entire paper carefully Slide 16 of 38

What to note? • Authors and research group – Need to know where to look for a paper on particular topic • Theme of the solution – Should be able to go back to the paper if you need more info • Approach to performance evaluation • Note any shortcomings Slide 17 of 38

So this paper is in print. . . • Be skeptical • If it sounds too good to be true, it often is …… Slide 18 of 38

More Advice … Slide 19 of 38

Three phases to reading • Determine if there is anything interesting at all in the paper. • Determine which portion of the paper contains the interesting stuff. • Should I read the whole paper? – Read the whole paper if necessary. Slide 20 of 38

Is there anything interesting? • Ideally, the abstract should tell you this, but frequently it does not. • Need to jump about – Read conclusion – Read introduction – Look at the bibliography – Glance at the TOC – Table of Content (if any) Slide 21 of 38

Which portion contains interesting stuff? • A paper typically outlines its organization at the end of the introduction. – Use this to determine which portion contains the exciting stuff. • Quickly scan the titles of each sections in the paper. Slide 22 of 38

Read the whole paper • Read with the following questions in mind – How can I use this stuff? – Does this really do what the author claims to do? – What if the assumptions and choices that the author made are discarded (or made invalid)? – Is source code available? – More questions …… Slide 23 of 38

Context and problem statement • What problems are the author trying to solve or trying to convince you of? • Are they important problems? – Why not? • What is the author’s thesis? Slide 24 of 38

Related work evaluation • Does the author describe other work in the field? • If so, how does this research differ from the other work? Slide 25 of 38

New idea • What new idea is the author proposing? – Architecture – Algorithm – Mechanism – Methodology – Perspective • Is the idea useful and practical? 26 Slide 26 of 38

What to evaluate? • What need to be evaluated to confirm the worthiness of the new idea? – Runtime – Throughput – Resource utilization – Model validation 27 Slide 27 of 38

How to evaluate? • How did the author go about conducting the evaluation? – Formalize and prove theorems – Run simulations – Artifact design and construction – Collect traces from existing systems Slide 28 of 38

Was the evaluation correct and adequate? • How was the data collection done? • Do you agree with the analysis of data? • Do you agree with the conclusions about the data? • Do you have any new interpretation of the data? • Can you suggest new ways to evaluate the data? Slide 29 of 38

Assumptions, drawbacks and extensions • Can you think of other aspects of the idea that need to be evaluated? • Can you think of extensions or modifications to the idea to improve it? • How would you evaluate your improvement? Slide 30 of 38

Assumptions, drawbacks and extensions • Can you apply the idea or method of evaluation to your own project? • Do the authors make any assumptions that are not valid or realistic? • Can you come up with a more general solution that does not rely on one or more of the assumptions? Slide 31 of 38

Future work • Does the author indicate how the work should be followed up on? • Does the paper generate new ideas? • Does the paper implicitly or explicitly provide a new way of doing other things or of thinking about problems? Slide 32 of 38

Summary If you remember nothing else: • Where to search papers • Scan papers • Read papers with questions in mind Slide 33 of 38

Download the presentation slides http: //www. slideshare. net/xqin 74 Google: slideshare Xiao Qin Slide 34 of 38

Further Research http: //www. eng. auburn. edu/~xqin Slide 35 of 38

My webpage http: //www. eng. auburn. edu/~xqin Slide 36 of 38

Download Slides at slideshare http: //www. slideshare. net/xqin 74 Slide 37 of 38

Questions Slide 38 of 38