edit for tact: If you're an academic who is more concerned with spreading knowledge than "publish or perish", please let the take away here be that you will be more successful at communicating ideas to the people who need them if you eschew excessive formalism and traditional obfuscation. I'm tired of wading through hundreds of papers that seemingly exist only to pad out CVs and plump up paywalls.
@aeva seems like an unnecessarily mean-spirited dunk on academics, yeesh, but regardless, I don't quite get it -- how does appending "we present" achieve your (I presume) goal of avoiding academic writing? there's no shortage of verbose and inscrutable papers that start off that way.
@topher_batty appending -"we present" not "we present"! The hyphen is important because it eliminates virtually all academic papers from searches results. Consider the reverse though: if you only want to read academic papers, you can remove that hyphen, and bam, only papers and paywalls.
@topher_batty I'm annoyed that when researching certain topics that are very hot for academic publishing, search results tend to be clogged with dense papers that are intentionally written to be as inscrutable as possible. I gather there's people whose whole jobs are to write as many of these as they can. The purpose of this seems to be something other than communicating useful ideas, because there's much clearer ways ideas like these can be expressed.
@topher_batty do try to be a little thick skinned. I'm in a bad mood today because my body is falling apart and I've read hundreds of We Presents and Novel Methods that omit important details, and it's largely led to a great deal of wasted effort.
@aeva IME the biggest factor is that academics are often simply bad at writing. The eternal style guidance by journals and educators boils down to "clarity above all else," but many brilliant computer scientists have no clue how to convey a series of thoughts in a way that makes sense.
Though strict length limits on paper submissions can hurt additionally. I've gotten "your definition could use an example" in reviews, oddly they never seem to say what should be cut in exchange. 😏
@julian@topher_batty maybe so, but it seems unlikely that such a deeply rooted cultural problem will be solved in a time frame anyone can be proud of, and so it's up to the rest of us to route around it.
@julian@aeva tangentially, this reminds me that last week on a whim I pushed a chunk of one of my papers through an LLM asking it to rephrase it in the style of a gamedev blogger (and then also in the style of James Joyce, and of a historic European explorer). Amusing results, but also not a particularly effective solution!
I would have more respect for the medium of academic computer science publishing if it was more common to see papers to the effect of "we tried this thing that sounded like a really cool idea, and let's be honest you thought about it too, but--surprise--it turned out to be absolute dogshit. Just a complete disaster. Total waste of time. Calling this a slideshow would be generous.", but instead usually people just fluff that sort of thing up so reviewer 2 can't tell it didn't quite work out.
@aeva from reading in other science related articles/blogs (e.g. In the Pipeline discussing academic chemistry papers) this is a problem with academia as a whole.
Journals apparently don't like papers that consist of, say "1001 chemicals that looked like a new painkiller but didn't work"
@aeva when I was working on my PhD, I joked that I wanted to make a conference or journal of negative results
But, of course, it wasn’t really a joke, I think it’d be great to have a venue for “we tried this, here’s what we did, it failed miserably, so here’s the path you should avoid if you’re interested in also working on this”
@aeva I mean, it’d have to be a surprising failure to be publishable, and unfortunately if you’ve done your job well in explaining the failure it probably makes it look obvious (which is already enough of a problem sometimes with reviewers for positive results, if you explain things so clearly that they then assume it isn’t interesting enough to be “novel”)
However, if the most obvious path doesn’t work for some non-obvious reason it’d be nice to warn others and get some credit for it before they make the same mistake
@tess I came up with a neat theory, but then I disproved it with this experiment should totally be publishable!
It’s like two papers in one! I came up with a theory that fires all these things. I disproved that theory. @aeva
@aeva a friend of mine spent a week or so going down a rabbit hole of a novel parameterised data compression technique which in the end turned out to be optimal with a set of parameters that made it behave identically to an existing well-known technique, which was pretty cool. essentially an accidental derivation from alternate principles. but, of course, not considered publishable.
@aeva Chemistry has been fighting with that problem for decades, if not longer. It is hard to get negative results published, and even if you do, you aren't going to get cited, as what will people say?
"We didn't use the reaction described in citation 18, as we read the paper and found out it didn't work."
idk, I think there's an ample space for retreading on the failures of others and finding interesting new information, or at least confirming empiricism.
@Canageek@aeva it's equal parts amusing and distressing how well that meme applies amyloid beta studies and seeing Derek's increasing levels of frustration about the literal billions of dollars it's wasted over the years
It's totally possible to be honest! I saw a SIGGRAPH talk titled "Learning From Failure" once, and let me tell you, there was a lot of failure in that talk and it was a great time.
@aeva project management papers are what you want to read then. They're probably still busy analyzing what went wrong in Heathrow Terminal 5 over a decade ago. But that's okay. They can write freely about failure because nobody in the real world reads any management research
@aeva well that's the problem with computer science - or it's real world application - nobody reads management papers (nah, mate. I got a 2 day Scrum certification!) Besides, reading papers about failure is always enlightening (and often entertaining too). If connect the dots, you'll find a lot of the same crap goes wrong in our industry as well...
@kwramm if I were to read a management journal, would I realistically find anything documenting what to expect if one were to, say, as a hobby project, try to render signed distance fields as variable density point clouds using a fast screen space voronoi approximation to fill the gaps?
@aeva I once went to a thing about people's tech failure stories with the rule that you couldn't attribute what you heard to the person/company telling the story
Dang. I apparently came up with my hack for removing academic articles from google searches on the very last day it would work, because, uh, wow. I can't believe google finally took search out behind the barn.
@aeva Wait does - seriously not work in Google search anymore??
I was just thinking about trying out this Kagi thing, and then I saw that they're So Excited about AI, but now I don't even know what to do. Resigning myself to "Bing with a condom" just doesn't feel great, especially because it's not like they'll be better than "Google with a condom" about anything, and it's not like the condom-providers can actually help with quality?
@fl0_id@aeva This is reminding me a bit of searx, which seemed cool, but even aside from the "you're just getting an anonymized amalgam of existing search from various places, with all its problems" aspect, any given searx instance would either become unresponsive or get blacklisted by various actual search providers like half the time, and I don't have the patience to run my own instance of anything.
Which I guess is to ask, does stract improve on any of this? What's it using under the hood?
@klara@aeva I haven’t tested it sufficiently, but it is not decentralized, probably exactly for these reasons and the suggest not everyone run their own crawler. They do have their own crawler, so it’s not just an amalgamation of bing or google. They have f e optics which are similar to kagi lenses to tailor your search. While centralized is a failure point it makes sense for this, and at least code-wise they are more open then kagi.
@klara@aeva it’s written in rust. I also don’t know the die of their current index or things like that. I might write to them to inquire about metrics/docs, governance and funding
@klara this is a problem where the only real solutions must have paranoid queers in a basement continuously downloading the entire internet energy. it's a lot of work to do it right
@aeva I've seen a few conferences/workshops that have started using a model of "you submit a paper minus the evaluation but with a detailed methodology, we accept or reject it, and then you carry out the experiments and we publish it regardless of the results" which is quite interesting. Dont know if its enough to solve this though.
@aeva publishing failures is an idea not without merits, but realistically if the only thing you change is (somehow) removing the incentive to publish mostly things that work well / better, many of the other issues you mentioned would be worsened. The space of "stuff that was a wrong/foolish/flawed idea" is quite large, making it even easier to flood the literature with half-assed, poorly written, unnecessary papers to dig through. Careful what you wish for.
@aeva can most academics be more concerned about spreading knowledge than publish-or-perish, though? The ones I know have their livelihood hanging on a rope because of the publish-or-perish mechanism, and for how much they love their field of study, they, well, love more having food and a roof above their head :-/
(not saying this is true for everybody, clearly, but I fear this is the situation for most people working in academia rn)
Add comment