14

I was given a paper to build off of for engineering research. The paper was published in a high impact journal but happens to be written very sloppily in the following sense.

  1. The proof of a central result of the paper cites a source which does not have the results that they claim. I found the correct paper which has results that do prove what they claim for their specific case (after writing the proof myself), but the way they wrote their proof heavily suggests that they thought the incorrect paper they cited proved their result. The proof I wrote from the paper I found is drastically different and requires conditions mentioned nowhere in their proof.

  2. They make the claim that their problem can be converted to a linear program. The linear program is very complicated (it involves quadruple and double sums, and a lot of variables), which I believe led the reviewer to be lazy and avoid seriously scrutinizing it. At first I was unsure as to whether there were gaps in my knowledge regarding the setup of the linear program, but after speaking with my advisor he also agreed that as written the linear program does not make sense. I attempted to look later in the paper in the hopes that they rewrote the linear program for their examples but it was not used.

  3. My advisor and I tried looking through their arXiv version in the hopes that maybe the mistake wasn't made there. The linear program they wrote in their arXiv version had even more inconsistencies.

  4. I tried reading through papers that cite this paper (of which there are 70), and everyone either cites their arguments as true or cites the paper because of its contribution.

  5. There is no explanation for the linear program as their "proof" just claims "these equations characterize the variables in the linear program."

Is this worth contacting the journal over? What can be done in this situation?

Improve this question
4
  • 12
    What outcome would you like to achieve?
    – The Doctor
    Commented Jul 9 at 17:35
  • At the end of the day, I just want to understand if the result is actually true.
    – AnotherPerson
    Commented Jul 9 at 18:45
  • 1
    By 'report', do you mean 'report to the authors', the journal editor, or the community?
    – smci
    Commented Jul 10 at 10:19
  • 1
    "At the end of the day, I just want to understand if the result is actually true." I am not sure that reporting mistakes or contacting the journal will help your understanding. Also, inconsistencies are not always mistakes, it could just be that, inconsistencies...
    – yarchik
    Commented Jul 11 at 15:27

3 Answers 3

Reset to default
24

This does sound serious enough that some sort of correction or at least awareness of issues should be in the literature. I don't know where that line is, but this sounds well over it. That said, I would also see what your advisor thinks of this. And before I contact the journal, I would contact the authors. This is a courtesy that some people appreciate.

Improve this answer
4
  • 18
    +1 First contact the authors. Not the journal, not the editors.
    – GEdgar
    Commented Jul 9 at 18:28
  • 1
    I appreciate the courtesy of communication with the authors; However, the authors may be less likely than journal editors to give the issue the attention it warrants, or to issue a retraction or correction where needed, This is especially true in the case of potential manipulation of results, as appears may be the case here given the clear inconsistencies. In any case the authors would be invited to respond, and it must be considered whether a neutral arbiter is best suited to handle the question.
    – Eric
    Commented Jul 9 at 22:17
  • 8
    @Eric If the authors don't respond, or respond poorly, then one goes to the editors. I didn't say to only contact the authors. This is a courtesy to them, not anything that gives them a veto. (Also, I don't think there's potentially manipulation of results given OP's description. It sounds much more like extreme sloppiness.)
    – JoshuaZ
    Commented Jul 9 at 23:11
  • 2
    @JoshuaZ Understood and agreed. However, should the need arise to raise a report to the editors, this becomes more difficult to do confidentially after having contacted the authors directly. If an open exchange is clearly the goal, then by all means, contact the authors and have that conversation. But where unclear, this can become fraught as noted, and the editors may be better equipped to make that determination fairly and without risk. In the end, I would defer to guidance from the COPE as noted below. Great discussion though, and by no means "one-size fits all".
    – Eric
    Commented Jul 10 at 0:16
8

There is also the option of publishing a comment with the journal, if the paper is reasonably recent.

Here are two detailed HowTos from quality journals, Nature and PRL.

https://www.nature.com/nature/for-authors/matters-arising https://journals.aps.org/prl/authors/comments-physical-review-letters

Both journals encourage contacting the authors beforehand.

Improve this answer
2

Anytime a reader finds errors or unfounded assertions that affect the findings of the paper, I would encourage them to contact the editors. Especially in an age of Generative AI, this vigilance is more important than ever to ensure that bad science and unfounded assertions are not perpetuated. This is precisely the purpose of scholarly review, both pre- and post-publication. The [Committee on Publication Ethics][1][2][3] has some great information in this regard.

There should be contact information for the editorial team on the journal website, or at a minimum some indication of leadership and their institutional affiliations. You can submit your findings to the editors, and they can investigate further if they deem it warranted, and potentially contact the authors for their response. Then they can choose whether and how best to correct the record, by inviting an author reply or by issuing a correctional statement (corrigendum) or retraction. Any legitimate journal editor would handle such a report fairly and the investigation confidentially and respectfully to all parties, and they would most likely welcome the effort to maintain their standards.

It sounds like you have done some due diligence and can support your report with objective evidence. That should be sufficient to make a detailed report or just a post-publication critique. How the journal handles this will be indicative of the quality of the journal itself.

[1]: Committee on Publication Ethics. (2023). Dealing with concerns about the integrity of published research. https://doi.org/10.24318/9cdeTy2t

[2]: Committee on Publication Ethics. (2024). Correction, retraction, or expression of concern? https://publicationethics.org/case/when-correction-retraction-and-expression-concern-could-be-considered

[3]: Committee on Publication Ethics. (n.d.) Handling of formally submitted post-publication critiques. https://doi.org/10.24318/o1VgCAih

Improve this answer
2
  • 10
    I would advise against this. This is an unnecessary and adversarial first step. This does not sound like a case of fraud or misconduct, but rather an oversight/sloppy mistake. Contacting the authors first and allowing them an opportunity to rectify the situation whether it be issuing a correction, or explaining to the OP why they feel it is correct is the course of action that embodies good faith. Much of the functioning of the academic community relies on good faith practices. Your option is the equivalent of a student not liking their grade and immediately escalating to the Dean or Provost.
    – R1NaNo
    Commented Jul 10 at 0:55
  • I disagree that this need be adversarial in any sense. The critique could be made with the assumption of good faith, and the editors could look into the matter, also with the assumption of good faith, giving the author the opportunity to explain, correct, retract or even let stand. What you describe is exactly what the editor would do. The only difference is the potential for preservation of anonymity in the process (should that be required) to ensure objective evaluation and consideration, just as we have with peer review, in most cases, and with letters to the editor and author replies.
    – Eric
    Commented Jul 10 at 11:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .