In retrospect, outsourcing the evaluation for hiring and promotion of every university professor to unregulated, nontransparent, unaccountable academic journals was a mistake. #academia#tenure#peerreview
US’ #NIH Australia’s #ARC : 'NO’ to ChatGPT for peer-review
US' #NSF and Europe's #ERC mulling it over with working groups.
Concerns:
*Privacy/Piracy “the information becomes part of its training data. ” (why I don’t chatGPT though I really hate writing)
*Error ”AI-written reviews will be error-prone"
*Bias ”against non-mainstream views”
*Boring "lack … creativity “
Here's a template for replying to review requests, if you like:
> If the article is available as a preprint, I will happily publicly review it (on PREReview) and then transfer my review to [your system].
>
> It doesn't look like the article is available as a preprint yet.
>
> If there is a credible reason why this article cannot be shared as a pre-print, let me know, and I will reconsider.
I'd go further than @AndrewReid's reply (https://neuromatch.social/@AndrewReid/110743035569990627)
and say that submitting unfinished work for peer review is borderline unethical under our current system: if the contributions of a peer reviewer takes a work from "unfinished" to "finished", then they're an author and should be treated/rewarded as such
And I would argue that it should be standard practice to update preprints following any changes that occur during peer review, such that all copies available online are identical (save for journal aesthetics).
I don't think that this is the norm yet[1], but I think we should be striving for that.
“Reviewers for a paper should be restricted in the number of citations that they can suggest from their own work (we suggest a 3 citation maximum, but a zero tolerance policy could also be justified).”
"Suggestions for improving this standard are welcome. They should be sent to the National Information Standards Organization…via email at <nisohq@niso.org>."
Double-blind peer review was never truly blind. "Experienced researchers can often correctly guess from which [person or] research group an anonymous submission originates." Now AI can improve the accuracy of those guesses. "Our method achieves an unprecedented authorship attribution accuracy, where up to 73% of papers are attributed correctly." https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0287611
Among the suggestions here: #authors should openly share #data, #code, & #preprints. #Journals should require it. Indexing services should reward the journals that do.
To me, what's new & interesting here is going beyond authors & journals to #indexing & #abstracting#services to create incentives for journals to improve their policies.
Auditory list really got treated to it's very own "I'm not gonna apologize for being successful" and "read John McWhorter, the absolver of white guys" and "I truly believe anyone can be successful if they just work hard enough" in the recent discussion about #PeerReview and #PrePrints
I love programming and thinking and talking about thinking. I have an education (BS, MS, PhD) focused on artificial intelligence and neuroscience.
I'm an advocate of the public academic pursuit of knowledge, the scientific process, peer review, and I see open source software and hardware as an essential part of the scientific process.
I see software user rights, including security and privacy, to be protected mainly by free and open source software.
I see the democratizing effects of the Internet, including distributed journalism and social networking, to be largely the effect of the collaborative development of free and open source software.
I am interested in free and open source manufacturing, including open source 3D printers and CNC machines. I believe open source manufacturing will be important for distributed manufacturing, allowing local manufacturing and local labor.
I see worker-owned coops as the way to safely transition from a non-democratic authoritarian top-down power structure of a traditional corporation to a democratic work environment, where the workers own the company and elect the board of directors, transitioning to democracy in the workplace.
I believe that socialism is a regulatory response to capitalism.
I believe that laws, money, corporations, and government are social agreements, and I'm in favor of democratic social agreements.
I believe in the organized non-violent boycott as a way to control capitalists and change corrupt systems.
I try to eat plant-based / vegan foods to boycott the animal industry, to help with the climate crisis, to improve my health, to avoid animal cruelty, and to avoid the extinction of species of plants, animals and ecosystems.
I have been diagnosed with Retinitus Pigmentosa, which is a disease of progressive retinal degeneration. I am legally blind, although I have about 5-degrees of vision remaining in my fovea. I'm interested in researching and developing BCIs (Brain-Computer Interfaces), specifically BCIs that function as vision prostheses that may help with conditions like RP, or the more common degenerative retinal disease AMD (Age-related Macular Degeneration).
I enjoy playing computer games like Age of Empires and Rimworld. I used to program computer games when I was younger and would like to get back to it one day.
I love playing music, especially bass guitar. I've been listening to a lot of Rage Against the Machine and Enya recently.
I enjoy reading books, mostly non-fiction.
I enjoy studying religions. I've found a lot of value in Buddhism, and I meditate often daily.
Nina and I have recently had our first baby, a boy we named Tyoma.
I'm currently working at Apple on the Vision Pro headset team.
Bis zum 18. August 2023 wird das Working Paper nun im Open Public Peer Review begutachtet: Alle Interessierten sind herzlich dazu eingeladen, ihre Rückmeldungen, Anregungen und Fragen in öffentlichen Kommentaren an Autorinnen und Redaktion heranzutragen. #peerreview#openscience
Micah Altman and I have a letter in Science: "Evaluating Peer Review at NIH" (complaining that #NIH's plan to improve its peer review process includes no assessment plan or mechanism, and suggesting they need to fund more stuff like that.) Paywalled: https://www.science.org/doi/10.1126/science.adi1570 #peerreview
Another #linkrot pattern that I've observed quite frequently in research is material hosted on university "personal page hierarchies". If I see a link in a paper or on a discussion that's of the form university.edu/ilastname/project then there's an 80% chance it's dead.
Lots of attention in #OpenResearch gets paid to people not posting their code at all; seems like the "university free hosting plan" problem gets disproportionately ignored, IMO....
@n8
> Another LinkRot pattern that I've observed quite frequently in research is material hosted on university "personal page hierarchies". If I see a link in a paper
Referencing any web link in a draft paper that isn't relatively permanent (eg DOI, web.archive.org, webcitation.org, archive.is) ought to be brought up as an issue during peer review.
"For the moment, we recommend that if #LLMs are used to write scholarly reviews, reviewers should disclose their use and accept full responsibility for their reports’ accuracy, tone, reasoning and originality."
PS: "For the moment" these tools can help reviewers string words together, not judge quality. We have good reasons to seek evaluative comments from human experts.
Apart from #quality, one concern is #confidentiality. If grant proposals become part of a tool's training data, there's no telling (in the NIH's words) “where data are being sent, saved, viewed, or used in the future.”