Hi! I was asked to peer review an article and this is my first one!
This is a well-supported and well-written article, but it failed to address a limitation that I feel as an expert is fairly significant. (It emphasizes the importance of a ‘write in’ option when collecting a demographic data but does not address data analysis limitations when there is not standardization.) This is actually something that MY research heavily focused on as a consideration to this field, and this anonymous author nominated me to be a peer review so they must know my research.
Is this an appropriate thing to leave feedback on? I hate to make them re-work their article, but this omission seems striking.
How critical should a peer reviewer be?
Thank you!
Comments
Call it as you see it. Explain it like you’d want to read it. Stay pro and get to the point.
The point of peer review is to provide valuable feedback (aside from determining publication). I think this point is definitely worth including and likely generally would just be a few paragraph revisions if I’m understanding correctly.
(in reality, they may or may not choose to address this criticism further, that is also their choice as the author).
Like you said, you’re an expert so flex that expertise!
Provide all the feedback you need to provide. I would suggest not thinking about it as criticism, but rather providing the feedback needed to get it to publication. If it simply cannot reach a state for publication, reject and explain the fatal flaws. If you see a path forward for it to happen contingent on completing certain revisions, then that’s what you should focus on.
You should definitely point out omissions so they can be included and make the paper stronger.
However do not be mean. Be constructive. I have read some absolutely mean reviews. That is not helping anyone. I always appreciate constructive suggestions. You said overall it’s a well done article! Tell them that!
One point – I do find that new reviewers tend to be more critical almost in a means to prove themselves. I view my job as an editor and peer reviewer to hopefully be helpful. It sounds like you have some good comments that could help the work so I’d go into it with the idea of being helpful, rather than thinking of it as being critical.
Don’t feel bad providing feedback if you think it actually addresses a weakness in the study. The authors could incorporate your suggestions to strengthen their argument or can at least clarify why it is not possible. Ultimately it’s up to the editor what happens. They can still accept a paper with highly negative feedback.
Praise it with great praise!
If you think a paper should be published, praise it. Make sure the editor knows it is good, and the authors have the ammunition they need to fight by the inevitable rejection.
After you have praised it, say how the paper should be improved. If that could be done just by text changes, make sure to explicitly say it could be done by text changes.
Be kind first, constructive second, mean never.
Consider whether or not their omission is made by other publications in the field, whether it breaks scientific norms or invalidates their study. Are their results or conclusions severely flawed or outright incorrect because of this?
As a reviewer I definitely tend to be lenient. If a work is *correct* and the level of novelty meets the usual standards of the journal, etc., then maybe I will express my concerns as feedback but approve.
Definitely mention it. It sounds like that might be addressed by simply adding a few sentences about limitations, but if it undermines the conclusions of the study that would be even more important to add.
Your job as a reviewer is To identity problems with the article. It’s not to push your personal agendas which can feel like a fine line.
If you think this is a major issue that’s ok to mention. Does it kill the paper or can the adjust formit? Is it enough to aknowledge this as a limitation? Be thoughtful in your reply and consider how your field at large would view it.
How about using the SII framework for assessment?
S – Strengths- what the article does well
I – Areas of improvement (obvious)
I – Insights – what aha moments or insights the article raises and gives rise to new ideas or meanings
This way you practice a great skill of providing meaningful assessment and also get your points across.
I really like this approach: https://aisel.aisnet.org/misq/vol40/iss3/2/
Valid review comment: this manuscript needs to address the limitations of approach X, specifically related to data analysis and non-standardized input. If needed, you can cite some references to support this, but do not just cite a bunch of your own work.
Ridiculous review comment: the authors show the limitations of their underlying knowledge by failing to acknowledge the basic principles of non-standardized input and its limitations for data analysis. They did not cite basic research that everyone should be familiar with <insert 5 of their own papers here>
You can be critical and not mean, demeaning or condescending. Your goal is to make sure the research is accurate, and publishable. You are not a rubber stamp, however it is also not a license to be an asshole about it.
I would simply delete the assertion of my own expertise.
You should be critical but kind. Certainly point out any limitations that they have not addressed.
I think you could just ask them to address your concern or elaborate on how they tried to handle it already. You can make it clear that the paper is already close to being ready to publish while also laying out a blueprint for improvement
“How critical should a peer reviewer be?”
It depends. Are you Reviewer #2?
You probably have a point but remember no one cares about your research topic in the whole world as much as you. Is it likely that readers would pick up on the omission you’ve found? If not then don’t make it a deal breaker
Point it out nicely, offer suggestions, don’t cite your own work (or they WILL know it’s you), and most importantly don’t be a pompous ass about it.
No research is or will ever be perfect. It doesn’t sound like this invalidates the entire study, so suggest that the authors mention this omission as a limitation to address in future studies.
I try to be strict when it comes to rigor, but less so when it comes to the impact. The paper has to be correct, but its better to have information out there so people can use it, rather than gate keep it. I always try to provide ideas how to arrive at the conclusion in the most resource efficient way, and request clarifications to make the paper as useful to reviewer as possible (e.g. provide a control comparing your system to a benchmark, as opposed to having an absolute value that cannot be put into a context). I would definitely not get angry or try to slow someone down if the paper is technically correct.
Thank you so much, everyone! I really appreciate the feedback, insight, etc.
I was honestly really honored to be asked to do this and I’m super excited to contribute to my field of research in this role. I definitely will not be rude or pompous, and I’m reviewer #1 (since someone mentioned which number I am lol).
Thanks!!