Review Process How to review Fausto Giunchiglia By

Review Process - How to review Fausto Giunchiglia By Fausto Giunchiglia and Alessandro Tomasi

Index: 1. Review Form 1 2. Review Form 2 3. Answer to the Reviews 4. Review Process

1. Review Form 1

1. Review Form 1 REVIEWER CODE: 1) Title 2) Author(s) 3) Paper Summary [short description of the message and maybe of how it has been developed. What if more than one message or no message? ]

1. Review Form 1 4) Type of Paper/Research described [Chose one of the options below and explain] - Research (with original results) of which kind (Theory, expemerimental, . . . ) - Application (case study, . . . ) - Synthesis of recent advances - Other. Please specify

1. Review Form 1 5) General Ratings [Rate within Bad/Weak/Fair/Good/Excellent] [0/1/2/3/4/5] Put the score and some text motivating your score about: 5 a) Relevance (with respect to the reference community) 5 b) Originality (incremental, new work, . . . ) 5 c) Significance of the work (how big the gap from the state of the art) 3 d) Technical soundness 3 e) References 3 f) Presentation

1. Review Form 1 6) Technical Soundness [Chose one of the options below and some text motivating the choice] - Technically correct - Minor errors (indicate them) - Major errors (indicate them) - Unsupported claims (provide a detailed explanation)

1. Review Form 1 7) Presentation [Rate within Yes/Somewhat/No] Put the score and some text motivating your score 7 a) Are the title and abstract appropriate? 7 b) Is the paper well-organized (discuss course and fine grained structure)? 7 c) Is the paper easy to read and understand? 7 d) Are figures/tables/illustrations sufficient? 7 e) Is the English acceptable? 7 f) Is the paper free of typographical/grammatical errors? 7 g) Is the reference section complete?

1. Review Form 1 8) General Recommendation [Please, chose one of the options below] - Very strong accept (beautiful paper!) - Strong accept (excellent and important contribution) - Weak accept (good paper, some new interesting ideas) - Weak reject (marginal, weak content, would require a major revision) - Strong reject (unreadable, nothing new, premature, contains major errors)

1. Review Form 1 9) Main Reason for your Decision [For accept choices please indicate one of the options below] - accept because of the originality (good ideas, sound presentation) - accept because of the quality of the proposed synthesis (useful review on recent advances) - other [For reject choices please indicate one of the options below] - reject because it is not relevant for the conference - reject because of the presentation (unreadable, unstructured) - reject because the content is too premature for really making sense - reject because of the lack of originality (results already known, or similar overview already published) - reject because of major errors

1. Review Form 1 10) Your Level of Expertise (Compared to Level of Others) - I am an expert of the field and know the relevant literature - I understand the problem, I know some of the state of the art - I only have a superficial understandings of the issues 11) Does the paper qualify for the best paper award [Y/N] 12) Comments to the Author(s) [Please, provide here a clear justification of your ratings, in particular with regards to the overall recommendation]

1. Review Form 1 13) Additional Comments to the Author (after circulation of reviews among reviewers) (Can be empty, cannot change previous review) 14) Additional Comments as Answer to Author's Answer and Modifications (Can Be Very Short: - Evaluation Of Author Answer - Value Judgement - Final Score (Possibly Changed))

2. Review Form 2

2. Review Form 2 REVIEWER CODE: 1) Title 2) Author(s)

2. Review Form 2 3) Main Message: Relevance: How relevant is the paper to the workshop? 0: not relevant at all 1: rather not relevant 2: relevant 3: very relevant Technical Quality: What is the technical quality of the paper? 0: really bad 1: bad 2: good 3: really good

2. Review Form 2 Presentation: What is the overall presentation of the paper? 0: really bad 1: bad 2: good 3: really good Overall Ranking: What is your overall recommendation? 0: strong reject 1: reject 2: weak reject 3: weak accept 4: accept 5: strong accept

2. Review Form 2 Confidence: Reviewer's expertise in the area 0: I know little about this area 1: I know enough about this area 2: I have good expertise in this area Why to accept? What are the most important reasons to accept this paper? (1 -3 sentences) Why to not accept? What are the most important reasons NOT to accept this paper? (1 -3 sentences) Comments: Detailed comments on the paper (primarily for the authors)

3. Answer to the Reviews

3. Answer to the Reviews <Brief introduction> LIST OF { <general comment quoted from reviews> <your answer arguing how you have accordingly modified the paper> } Moving now to the more specific comments: LIST OF { <specific comment quoted from reviews> <your answer arguing how you have accordingly modified the paper, providing detail but not too much> } <Concluding sentence>

4. Review Process

4. Review Process 0) Abstract (send it to tomasi@dit. unitn. it not later than a week before the presentation) 1) Presentations (Fortunate situation where you may know of what the paper is about) 2) All Papers Submitted by June, 13 th 3) Review Allocation (by Program Chair) by June, 20 th 4) Reviews Due by July, 4 th 5) Reviews circulated to Reviewers for additional comments 6) Reviews send to Author by July, 11 th 7) Author sends back answer and modified paper by July, 23 rd 8) Reviewers provide final answer 9) Exam Pass/No Pass (of Authors and Reviewers) by July, 29 th

4. Notice!!! In the paper you have to use the same version of the stylefiles. It is available by downloading it from http: //www. springeronline. com/sgw/cda/frontpage/ 0, 11855, 3 -111 -2 -124365 -0, 00. html Avoid using stylefiles acquired from other sources as these may not be correct

How to do presentations 23

Structure 1. Presentation methods 2. Attitude towards the audience 3. How to select content 4. How to structure 5. Introduction 6. Stage fright 7. Main part 24

Structure 8. Using pictures 9. Visual aids 10. Interposed questions, interruptions 11. Final discussion 12. The end 13. Optical – acoustic factors 14. Ancillary conditions 25

1. Presentation methods Rule No. 1: Control effect Rule No. 2: Integrate audience 26

1. Presentation methods w Seminar w Lecture w Presentation with final discussion w Presentation with intermediate discussion 27

Comparison of some presentation methods What remains Time to prepare / h Open discussion 80% 4, 0 Presentation with Intermediate discussion 60% 2, 5 Presentation with Final discussion 30% 1, 7 Lecture 20% 1, 0 28

2. Attitude towards the audience w Who is the audience w What do they know w What are they interested in w What do they understand 29

2. Attitude towards the audience w w w Contact with eye Simple and understandable languages Concentrate on the important Credibility Reply to objections Avoid „techno“ language 30

3. How to select content How much? From where? What is interesting? What is the objective? What should the audience do? Do not want too much! 31

4. How to structure – catch words w Speak as you think – follow the thinking w Use written concept of catch words 32

5. Introduction w w w Try to find common understanding with the audience Lead to the subject Initially: create attention 33

5. Introduction Say at the beginning w The structure w The length w Invite to a dialogue But at the beginning: Who are you 34

attention Hope that it will end soon Length of presentation 35

6. Stage-fright Afraid to speak w w w Is natural! Everybody has it ! Is not forever! 36

Reduce Stage-fright w Be well prepared! w Learn important parts by hard! w Relax! w Look for a positive „point“ w Do some contacts before presentation! 37

7. Main part Introduction Main part w Current situation w Potential solution w How to realise End 39

8. Using pictures / figures Picture w w Take from the context of the audience Has to increase the message To explain the issue Does not be an end on itself 40

8. Metaphorical language w w w To be a comparison Should be practial Story Citation Joke 41

9. Visual aids w w w Black board Projector Film Videobeam Overhead-Display 42

Presentation programs w Grafical Layout w Colours – Contrast – Background w Clipart-files w Fonts (size) w Produce hand outs 43

What we keep 10% 20% 30% 50% 70% 90% Reading Hearing Seeing Hearing & Seeing Reporting Do it yourself 44

10. Questions w Question w Are welcome w Should be answered immediatly w Bit can also be delayed or forwarded to somebody else w Interupptions w ignore! w Look at the person! w Stopp speaking! 45

11. Final discussion w w w Announce already in the introduction Write up issues of discussion Keep sequence of questions Repeat questions (if necessary) Draw conclusions 46

12. End w Avoid w New questions which do not help in the conclusions w Main ideas w Should be summarized 47

12. End Main idea + review objective + how to realise = Good end 48

13. Acoustic factors w No „speaking smoking" w Speak loudly and slowly w You should vary volume and speed w Make breaks w Try presentation beforehand 49

13. Optical factors Behave natural! w w w With your body With your hands With your eyes 50

14. Other circumstanceas w Room size w Lighting w Ventilation w Sockets w Other means (Microfon) w Breaks 51

Reading - How to read Fausto Giunchiglia Literature: Bruno Buchberger, Thinking Speaking Writing By Fausto Giunchiglia and Alessandro Tomasi

Index: 1. The Role of Literature 2. What is literature 2. 1 Classify the results 3. The Documentation of Literature 4. Steps in the Use of Literature

1. The Role of Literature

1. The Role of Literature “Keep Re-Inventing the wheel” For many problems: • the solutions is already in the literature • the literature has solutions in similar problems

1. The Role of Literature It is necessary to know how the literature is organized: • within your research community • outside your research community [translate other concepts in our community] Necessary Useful but not Necessary

1. The Role of Literature There is an interleaving of: 1) Try to solve the problem yourself 2) Read the Literature

1. The Role of Literature Interactive Process of Reading Papers: Step 1 Quantity Step 2 Step 3 … Depth

1. The Role of Literature How do I know when I can stop reading? When I know the 90% of the papers cited by relevant papers

1. The Role of Literature To keep Scholarship: • go to conference • talk to people (best opportunity in conference) • journals • proceedings • web • …

2. What is Literature

2. What is Literature 1. Monographs / Books Main Way to do Career 2. Journal Articles 3. Articles in Collections 4. Papers in Proceedings of Conference 5. Papers in Proceedings of Workshop 6. Technical Reports 7. Grey Literature (Ex: Web Sites) More important than papers

2. What is Literature Each research community has: 1 -2 Top Journals 1 -2 Top Conference

2. 1 Classify the results

2. 1 Classify the results 1. Monographs / Books: Note: the research is finished Contents: extensive/complete description of research Originality: None (2/3 years old) Quality Control: Very Good

2. 1 Classify the results 2. Journal Articles: Note: Come regularly Contents: Very good, very crisp, specialized Originality: Full of two years ago (in CS is obsolete) Quality Control: Very High (Many shots)

2. 1 Classify the results 3. Articles in Collections: Note: Not regularly Contents: Very good, very crisp, specialized Originality: Full of two years ago (in CS is obsolete) Quality Control: Lower

2. 1 Classify the results 4. Conferences: Note: Is Refereed Contents: New results, almost complete, semi-final Originality: Good (6 months ago) Quality Control: Medium (only one shot: accept or reject)

2. 1 Classify the results 5. Workshop Note: Is Not Refereed Contents: New results, almost complete, semi-final Originality: Good (6 months ago) Quality Control: Medium (only one shot: accept or reject)

2. 1 Classify the results 6. Technical Reports Note: --Contents: Detailed about a specific topic Originality: Maybe Quality Control: Low (No Refereeing Process)

2. 1 Classify the results 7. Grey Literature (Ex: Web Sites) Note: The main way to publish our results Contents: Random Originality: Random Quality Control: Random

3. The Documentation of Literature

3. The Documentation of Literature • Library - author files - keywords files • Review Journals - Computing Journal - Artificial Intelligence Review - Scientific Magazines for Spreading Communications - Journal for Publish new Results • Bibliographies • Citation index • Research index

3. The Documentation of Literature Research Community are organized into societies Produce: Organized Conferences, Journals, Magazines Examples of Societies: ACM, IEEE, ECCAI, VLDB, …

3. The Documentation of Literature Citations Index: • defines journals • cites titles, authors, abstract of all paper Example: ISI, the most important for Science

3. The Documentation of Literature Research Index: • the new game • is on line • how many times you are cited in Internet Example: http: //citeseer. nj. nec. com

4. Steps in the Use of Literature

4. Steps in the Use of Literature 1. Use Cite. Seer (http: //citeseer. nj. nec. com) 2. Ask to Colleagues, Advisor, 3. Friends, … by e-Mail 3. Consult References 4. Consult Review Journals 5. Ask Authors