Publishing Papers and the Publication Process Andy King

Publishing Papers and the Publication Process Andy King http: //www. cs. kent. ac. uk/~amk Abstract: This talk will discuss scientific writing, the publication process and make suggestions on how to improve the quality of your scientific writing. Publishing examples will be drawn from the world of logic programming. Acknowledgements: Some of this material has been adapted from MIT AI Lab Working Paper 316 that is edited by David Chapman

Section 0 Structure of this talk z Why bother with papers? z Types of papers z Finding papers z Reading papers z Writing papers

Section 1 Why bother with papers? z Official reason: y. Enables physically distributed researchers to communicate z Unofficial reason: y. Peer-reviewing provides feedback if you do not get much local support (ie new anti-unification algorithm) y. Provides free travel and cheap holidays y 3 papers in good conferences/2 journal papers will make you practically unfailable (ie Ulf Nilsson thesis on “Abstract Interpretation and Abstract Machines”) y. Give you control over your future

Section 2 Types of papers

Origins of the scientific literature z Scientists used to write letters or circulate (privately published) books, treatises, pamphlets, etc z Literature formally began in 1665 with: y Philosophical Transactions of the Royal Society y Journal des Sçavans z The secretaries of the: y Royal Society of London y Academic dés Sciences would collect reports of their meetings and these “journals” proved to be very popular z “Science since Babylon” reports that the number of papers has been doubling every decade since 1700

Different types of scientific literature z Grey literature: y trs, pre-proceedings, web pages, manuals, etc. z Secondary literature: ybooks that are lightly refereed z Primary literature: y. Journals, monograms, proceedings y. Report specific, identifiable advancement in knowledge y. Checked by peer reviewing y. There is no substitute for the carefully prepared, thoroughly vetted paper, published in a reputable forum

Impact rating/estimating reputability z Using ln(cites per annum+1)/papers per annum: OSDI, PLDI, SIGCOMM, MOBICOM, ASPLOS, TOCS, SIGGRAPH, JAIR, SOSP, MICRO, POPL, PPOPP, Machine Learning, Computer Networks, Computational Linguistics, JSSPP, VVS, FPCA, LISP and Functional Programming, ICML, Data Mining and Knowledge Discovery, SID, ICSE ACM Transactions on Networking, OOPSLA/ECOOP, Workshop on Workstation Operating Systems, Journal of Computer Security, TOSEM, Workshop on Parallel and Distributed Debugging, Workshop on Hot Topics in Operating Systems, Journal of Cryptology, CSFW, ECOOP, Evolutionary Computation, TOPLAS, CAV, AAAI/IAAI, PODS, Artificial Intelligence, NOSSDAV, OOPSLA, IJCAI, , VLDB Journal, TODS, HPCA, LICS, JLP, ICCV, IEEE Real. Time Systems Symposium, KR, TISSEC, ACM Conference on Electronic Commerce, TOIS, PEPM, SIGMOD Conference, Formal Methods in System Design, Mobile Agents, REX Workshop, NMR, Computing Systems, LOPLAS, STOC, Distributed Computing, KDD, Symposium on Testing, Analysis, and Verification, Software Development Environments SDE, SIAM J Comput, CRYPTO, Multimedia Systems, ICFP, Lisp and Symbolic Computation, ECP, CHI, ISLP, ACM Symposium on User Interface Software and Technology, ESOP, ECCV, ACM Transactions on Graphics, CSCW, AOSE, ICCL, Journal of Functional Programming, RTSS, ECSCW, TOCHI, ISCA, SIGMETRICS/Performance, IWMM, JICSLP, Automatic Verification Methods for Finite State Systems, WWW, IEEE Transactions on Pattern Analysis and Machine Intelligence, AIPS, IEEE Transactions on Visualization and Computer Graphics, VLDB, Symposium on Computational Geometry, FOCS, ATAL, SODA, PPCP, AAAI, COLT, Information and Computation, Java Grande, ISMM, ICLP, Structure in Complexity Theory Conference, IEEE Transactions on Multimedia, Rules in Database Systems, ACL, CONCUR, SPAA, J Algorithms, DOOD, FSE, ICDT, Advances in Petri Nets, ICNP, SSD, INFOCOM, IEEE Symposium on Security and Privacy, Cognitive Science, TSE, Storage and Retrieval for Image and Video Databases SPIE, NACLP, SIGMETRICS, JACM, PODC, International Conference on Supercomputing, Fast Software Encryption, IEEE Visualization, SAS, TACS, International Journal of Computer Vision, JCSS, Algorithmica, TOCL, Information Hiding, Journal of Automated Reasoning, ECCV, , PCRCW, Journal of Logic and Computation, KDD Workshop, ML, ISSTA, … z See http: //citeseer. ist. psu. edu/impact. html

Section 3 Finding papers

Finding ancient papers z Raid your supervisors cabinet and talk to others z Ask the author for an off-print or report: y. R. Gluck, V. Pratt, C. Lassez, … z Use inter-library loan to request papers: y. D. Kohler, “Projections of Convex Polyhedral Sets”, Technical report ORC 67 -29, University of Berkeley.

Finding modern papers z Firing “Synergistic Analysis” at http: //citeseer. com: y. Citations to the paper itself complete with a graph y. Suggestions on related documents, bibtex, ps, pdf, … z Check http: //www. informatik. uni-trier. de/~ley/db: y. P. Schachte, “Precise goal-independent abstract interpretation of constraint logic programs”, Theor. Comput. Sci. , 293(3): 557 -577 (2003) z Register interests at http: //arxiv. org/corr/home

Finding non-existent papers z. Trend setting work has not yet appeared: y. Conference papers take 4 months to appear y. Journal papers take 2 years to be published z. Ideas get circulated prior to publication through drafts, visits, seminars z. Penetrate the so-called secret paper passing network by: y. Pass your drafts around y. Offering to help your supervisor with reviewing

Section 4 Reading papers

How do you read a research paper? z Read the introduction and related work z Carefully study any worked examples, checking the working by hand if possible z Read a paper looking for weaknesses and holes z Ask sceptical technical questions: y. Does the method scale-up to real problems? y. Can the technique be mechanised? y. Where is the construction ad hoc or informal? y. What are the interesting theoretical questions? z Most papers are broken

Reading foreign theses z Use http: //www. google. com/language_tools y. Translate text box or web page y. Divide thesis into chapters and check for accuracy « La plupart des systèmes critiques utilisés dans les domaines de l'avionique, du transport ou de l'énergie intègrent désormais des applications logicielles embarquées. Ces applications doivent être soigneusement validées afin d'offrir des garanties de sécurité et de sûreté de fonctionnement. Pourtant cette phase de validation repose encore sur des techniques de preuve et de test très artisanales. » “The majority of the critical systems used in the fields of the avionics, transport or energy integrate from now on embarked software applications. These applications must carefully be validated in order to offer guarantees of safety and reliability. However this phase of validation still rests on techniques of very artisanal proof and test. ” z Initiated a grant with Arnaud Gotlieb at Rennes

Different perceptions of the same paper z Inevitably someone will pass you a paper and say, “this is brilliant, you should read this” z But, to you, the paper is not that brilliant or relevant z This discrepancy occurs because: y. When the first person reads the paper, they have an idea which relates to your area that you miss: x. V. A. Saraswat et al, Semantic Foundations of Concurrent Constraint Programming, POPL 1991, 333 -352 x. E. Rodríguez-Carbonell and D. Kapur, An Abstract Interpretation Approach for Automatic Generation of Polynomial Invariants. SAS 2004, 280 -29 y. Reading papers somehow awakens latent ideas

Section 5 Writing papers

Is your work ready for publication? z Can you write a convincing introduction? y. List and explain your 3 contributions z Can you fill 8 LNCS pages with a novel core? Mandatory: y abstract, intro, y preliminaries, y related work, conclusions, y references. z Even if you decide after writing, that you work is too premature, then you have still advanced. z Deadlines drive research but are often missed

Targeting a conference z Study the topics list: “Contributions are sought in all areas of blah including but not restricted to …” z Many theory conferences solicit application papers: “Specific attention will be given to new applications of thingamajig” z Search the literature for related papers published in the conference series (work by case law) z Consider the quality of the symposia: “A total of 128 submissions were received of which 28 were selected for publication” z Deadlines are surprising flexible (ESOP)

How to improve a paper z Craft the introduction around the call for papers z Counter-examples can inflame the referee so: y Check that you “do not misrepresent their work” z Short papers are better than long papers: y « Je N'ai fait celle-ci plus longue que parceque je n'ai pas eu le loisir de la faire plus courte » y “I have only made this letter rather long because I have not had time to make it shorter”, Pascal, 1656. z Provide a worked example to support tricky theory z Temporarily add a proof appendix z Use a web interface to demo your ideas: yhttp: //www. cs. kent. ac. uk/~amk/susweb. html

Closing down your paper z Referees are often selected from the related work so be tactful: y“Recently, [7] have shown how Def formulae can be cleverly encoded as ACI-terms. The initial experimental results reported in [7] are promising though widening is required for the larger programs” z Use journal rather than conference versions: y. M. Bruynooghe, D. De Schreye, B. Krekels, “Compiling Control”, J. Logic Program. , 6(1&2): 135 -162, 1989 z Journal abbreviations, ie, “J. Logic Comput. ’’: yhttp: //wos 17. isiknowledge. com/help/N_abrvjt. html

What the sub-referee sees REVIEWER-NAME: Andy King PAPER-NUMBER: 14 B: I can accept this paper, but I will not champion it (accept, but could reject). CLASSIFICATION: Y: I am knowledgeable in the area, though not an expert; REVIEWER-EXPERTISE: 1: Research PAPER-TYPE: 4 - Summary of the paper: [32 lines] 5 - Comments for the committee only (Not shown to the author(s)): [10 lines] 6 - Comments for the author(s) - will be sent to the submitters: [44 lines] 7 - Points in favour or against 8 - Co-reviewer(s) for this paper: None

What the author receives from Cyber. Chair From: ESOP 2003 - Cyber. Chair [curtim@di. unipi. it] Subject: ESOP 2003 Notification Dear Andy King, We are pleased to inform you that your paper, titled "Goal-Independent Suspension Analysis of Logic Programs with Dynnamic Scheduling" has been accepted for presentation at the conference ESOP 2003. Actually, the committee selected only 25 papers out of 99 submissions. Please carefully take into account the enclosed comments by the reviewers when preparing the camera-ready version. It is incumbent upon you to do so. Your camera-ready paper, NOT exceeding 15 pages, is due on January 17, 2003, in the format specified by Springer-Vela at the URL http: //www. Springer. de/comp/lances/authors. html Sincerely, Pier Paolo Delano (ESOP 2003 PC Chair)

What the author receives from Cyber. Chair (part II) First reviewer's review: >>> Summary of the paper <<< [59 lines] >>> Comments <<< [47 lines] >>> Points in favour or against <<< [69 lines] Second reviewer's review: Third reviewer's review: [Note that the scores are not seen by the author with Cyber. Chair. Better conferences, such as POPL and ICLP, tend to use 3 or 4 referees]

What happens in the secret PC meeting? z The chair presents a ranking all the papers AAAB, …, DDDD (updated during meeting) z The list is top and tailed z The discussion focuses on the middle-ground and the number of conference streams. z Sometimes borderline papers are refereed again over night or over lunch, sometimes remotely. z The referees reports are tweaked to remove bias and offensive language z PC member papers are considered at the end

Productive paper writing z ICLP poster by Lunjin Lu led to: y. TPLP, ICLP, ESOP, SAS z RS visit of Samir Genaim led to: y. ESOP, TOPLAS and ESOP submissions z Seminar on Reunion led to: y. TPLP z 3 week visit can be as productive as working 3 months alone

Routes to a journal paper z Extend a workshop or conference paper with implementation details, proof of correctness, respond to reviewers comments (regular paper) z Submit to a conference with a journal route, for instance, ESOP in SCP (special issue) z Write a pearl, letter or technical note (fast track) z Make accessible the ideas your thesis (generalist)

Referee/author interaction (stage 1, 22/11/02) To: Bart Demoen Subject: Submission to TPLP Dear Bart, Please find enclosed as an attachment a copy of our paper “Computing Convex Hulls with a Linear Solver” for consideration as a pearl in TPLP. The paper is a revised and extended version of a paper that appeared in Logic-based Program Synthesis and Transformation (LOPSTR) in 1996. Best regards, Florence Benoy, Andy King and Fred Mesnard [Paper is 6 pages long but it takes a month to journalise a section of the original 20 -page LOPSTR paper since portable code has to be written, exposition added and add a new correctness result proven in computational geometry. Bart is the area editor. ]

Referee/author interaction (stage 2, 12/3/03) From: Bart Demoen Subject: TPLP/LN/5 Andy, Your paper TITLE: Computing Convex Hulls with a Linear Solver AUTHORS: Florence Benoy, Andy King and Fred Mesnard has been accepted conditionally for publication in the Pearls Section of TPLP. I will get back to you with one more referee report and shortly afterwards with the conditions for acceptance - they are related to some rewriting and polishing of the text. Regards, Bart

Referee/author interaction (stage 2 cont’d, 30/3/03) From: Bart Demoen Subject: TPLP/LN/5 Dear Andy and other authors of Computing Convex Hulls with a Linear Solver. Here additional comments of mine. . . my apologies that it took so long. Personally I tend to agree with referee 3 that there is no LP pearl in your paper. This is not related to what referee 4 refers to: # Whether it could be a logic programming pearl depends # on your view of the relationship between logic and constraint # programming, but I'm inclined to say yes. The actual CLP code must be improved and explained better. One referee complains very justifiably about the variable naming convention. That is one point that can be improved. Another point is… [262 lines on instructions on 4 referees reports of 98, 65, 92 and 59 lines]

Referee/author interaction (stage 3, 30/7/03) To: Bart Demoen Subject: TPLP/LN/5 Bart, Thanks for your meta-report on the referees' reports. We have revised the paper to address these comments -- see the enclosed attachment. In fact we provide responses to each of your comments so that you can check things off. Sorry that our response has been so slow. The new complexity counter-example in (which is now very simple) represents several weeks of thinking. Best regards, Andy, Florence and Fred > I would appreciate if you do not take the minimalist approach: some > comments of the referees should be addressed even if I do not mention them. We have expanded the proof of correctness and have now added intuition to make it more readable. [letter of response is 367 lines and paper is now 11 pages]

Referee/author interaction (stage 4, 26/9/03) From: Bart Demoen Subject: TPLP/LN/5 Hi Andy, Here are the comments of my harshest referee : -) (it is also the only referee at this point, because we are in the end phase) I have added my own comments prefixed by > I have to ask you for another iteration. If it is of any consolation: of all papers submitted/accepted by TPLP, pearls take most time and most iterations ( …); I think this iteration is not very demanding on you and you are very close. If you need more explanation on the comments, please let me know. Cheers, Bart [Bart’s new meta-report on the harshest referee’s new report is 124 lines]

Referee/author interaction (stage 5, 14/10/03) To: Bart Demoen Subject: TPLP/LN/5 Dear Bart, Please find enclosed a revised copy of our paper as an attachment. As before, a response to each of the issues you raised is given. Best regards, Florence, Andy and Fred. > > ignore the part where the referee complains about "this is not a pearl" > > the remark that a naive implementation (without copy_term and > > prepare_dump) would at first be easier to understand, is correct and > > would make the presentation better from the didactic point of view; > > for a pearl, that would be indeed very nice - can you do it ? We have followed this suggestion, restructuring the development of project in 3 stages. [New revision of paper is 12 pages and the new letter of response is 151 lines]

Referee/author interaction (stage 6, 17/10/03) From: Bart Demoen Subject: TPLPLN 5 Dear Andy, Florence, Fred, I am pleased to tell you that your nightmare is coming to an end: your paper is really in the final stage of acceptance, meaning that I will now bug you only with syntactical, grammatical, typographical and similar matters. Allow me to comment on your most recent changes: # We have followed this suggestion, restructuring the development of # project in 3 stages. I would suggest that you change the sentence "This leads to the following revision" on page 6 into "This leads to the following (SICStus Prolog specific) revision" or something in the same spirit. [142 line report on rewording, grammar and “political” comment]

Referee/author interaction (stage 7, 17/10/03) From: Bart Demoen Subject: TPLPLN 5 Dear authors of the TPLPLN 5 submission, I am very happy that your submission Computing Convex Hulls with a Linear Solver as a TPLP Programming Pearl is in the final stage of acceptance: I herewith recommend it to Maurice Bruynooghe - the editor-in-chief for publication in that particular section of TPLP. Maurice will send you additional requirements you should comply with for the final version: they relate to form and what exactly needs to be send to him as final copy. From now on, you deal with him. Thanks for submitting to the programming pearls section of TPLP: I can only wish we receive more submissions like yours. Regards, Bart Demoen

Things that you shouldn't do (but I’ve done) z. Send a paper to the wrong journal, for instance, Jo. Pa. DC submission rejected after 2 years z. Not respond to the referee’s comments z. Forget to update related work in the final copy, for instance, embarrassing erratum z. Don’t confuse your versions, for instance, JLP technical note

Pitfalls of writing z Postponing the introduction and conclusions yremove content-free generalities from the intro z Selling your work with superlatives, better to say y“MHC compares favourably with BDDs” z Perfectism y. Consider your paper to be a scratch-pad z Displacement activities such as yendlessly repolishing perfectly adequate text yspending too much time on the formatting ytweaking your bibliography/performance figures

Making the most of what you have got z Keep your web page up-to-date. If you add links to ps, then you are more likely to be cited. z Check what you have signed: y“The Author may publish his/her contribution on his/her personal Web page provided that he/she creates a link to the above mentioned volume of LNCS at the Springer-Verlag server or to the LNCS series homepage at http: //www. springer. de/comp/lncs/index. html” y. If copyright is a problem, then create an “almost identical to the journal” technical report version. y. Many insist on the Computing Research Repository (Co. RR) http: //xxx. lanl. gov/archive/cs/

The international “Who’s Who” of publishing Academic Cites D. Johnson 15846 J. Smith 7693 H. Zhang 6514 J. Ullman 13004 E. Clarke 7618 A. Pnueli 6462 A. Gupta 9930 L. Lamport 7605 A. Aho 6400 R. Milner 9711 J. Dongarra 7552 H. Garcia-Molina 6218 R. Rivest 9467 D. Knuth 7128 J. Hennessy 6184 S. Shenker 9019 L. Zhang 6942 R. Jain 6121 V. Jacobson 8408 R. Agrawal 6921 D. Goldberg 6097 M. Garey 8344 C. Papadimitriou 6704 … R. Tarjan 8144 R. Karp 6651 M. Bruynooghe S. Floyd 8141 R. Johnson 6566 P. Stuckey 1118 990 See http: //citeseer. ist. psu. edu/mostcited. html for the top 10000 cited authors of the 773109 cited authors in computer science. Find your ranking at http: //citeseer. ist. psu. edu/allcited. html