CSCI 7000-05 Spring 2026: Paper Critiques

SUMMARY

We will have two papers discussed per meeting. Each student need to read all assigned papers and write reports. However, if you are presenting in a given meeting, you need not write a report for any of the other papers.

Students should critique the papers to be discussed in class. Do not directly copy the abstract, introduction, conclusion or any other part of the paper - you must add value and insight beyond what is already in the paper. Unless specified otherwise, all paper critiques are due by 12pm the day of the respective class and should address the following questions (inspired from this excellent resource on how to Read an Engineering Research paper by William Griswold):

1. What I liked

What is your analysis of the identified problem, idea and evaluation? Is this a good idea? What are the most interesting points made? What are the most controversial ideas or points made? For work that has practical implications, you also want to ask: Is this really going to work, who would want it, what it will take to give it to them, and when might it become a reality?

2. What I disliked

What flaws do you perceive in the work?

3. Future directions for this research

Not only what future directions do the authors identify, but what ideas did you come up with while reading the paper? Sometimes these may be identified as shortcomings or other critiques in the current work. Examples include, but are not limited to: Can the same problem have a different solution? Can the same solution be applied to a different problem? Would a bigger evaluation be appropriate?

4. Open questions

What questions are you left with? What questions would you like to raise in an open discussion of the work? What do you find confusing or difficult to understand? By taking the time to list several, you will be forced to think more deeply about the work.

5. Most important for class discussion

What are you looking forward to get out of a class discussion based on this paper? What are the most important points you would like to discuss with colleagues and the professor in class?

6. Summary

What is your take-away message from this paper? Sum up the main implication of the paper from your perspective. This is useful for very quick review and refreshing your memory. It also forces you to try to identify the essence of the work.

OPTION TWO: Tool Experience Report

Some students gain more insights using experiential learning: read a paper, download the tool, and use it for your own usecase. Then write a Tool Experience Report -- instead of the Paper Critique format outlined above. After using the tool, write a 1-page, concise reflection on using the tool introduced by the research paper, following this format:

1. Your Experience Using the Tool

What specific task(s) did you perform with the tool? How easy or difficult was it to install, set up, and use? Did the tool behave as described in the paper? Mention any differences or surprises you found.

2. Strengths

What are the strongest aspects of this tool (e.g., accuracy, usability, speed, integration, insights)? What features or design choices did you find especially effective or well thought out?

3. Weaknesses

What limitations or challenges did you encounter (e.g., poor documentation, installation issues, narrow scope, inaccurate results)? Were there any tasks or contexts where the tool performed poorly or failed to meet expectations?

4. Potential for Practical Impact

How could this tool influence software engineering practice if adopted widely? What types of developers, teams, or companies would most benefit from it? What barriers to adoption (e.g., trust, policy, scalability) might stand in the way?

5. Reflection and Recommendations

What did you learn from using the tool that you wouldn’t have learned from reading the paper alone? What one improvement would you recommend to the tool’s creators?

Submission Procedure

Submit your paper critique by 12pm on the day of the class on Canvas. By sending your questions earlier than the class time, the presenter will have time to find some other relevant materials that are addressing your questions. Thus, the presenter will be able to better serve you.

Paper critique policies