A conversation that keeps popping up in my mind since FOSDEM centers around open source projects and “AI,” and I still don’t know what I think. So let me share some thoughts here on the famously nuance-friendly Internet. 😜
During a chat w/folks from several open source organizations, someone suggested GNOME could attract funding by “sprinkling some AI on it.” Several folks laughed at the topical joke, but then realized it was in earnest. 🧵
My gut reaction—and judging by the room, the reaction of others—was “hell naw.” But then the person clarified, and I understood what they were actually saying. And I think it mostly jives with my thoughts around “AI.”
The suggestion, as best I can describe it, was that there are a lot of opportunities for genuinely useful, responsible, offline, machine-learning-powered improvements to a platform like GNOME.
@evelyn there’s more to the thread, but think offline features around: translation, object recognition, camera processing, text to speech, speech to text/live transcription, search, autocorrect… hell, even automatic playlists in Music could be algorithmic and would 100% be branded as “AI” if in a major app today. :)
For example: object recognition in the Image Viewer app to remove backgrounds; algorithmically improved camera quality in video calls; autocorrect! These are all areas that use ML algorithms on other platforms, and I don’t think that’s bad; you take a bunch of data, train an algorithm, then ship that in the OS/GNOME/etc. to be genuinely helpful.
First, we can get on board with that, right? Personally, I don’t consider that “AI,” even.
So… how does that attract funding so that we can invest in the platform as a whole, making everything (not just these hypothetical ML-powered improvements) better—funding that could be put towards continuing to improve accessibility, developer APIs, research and user testing, and features that our users care about?
Well, you kind of have to pitch it as AI. 😬️ Let me explain.
Like it or not—and I absolutely hate it—“AI” is where the hype is now. I don’t want to play into it. I think “generative AI” is extraordinarily overblown, problematic, and burning an immeasurable amount of resources. So let’s not do that.
But if you could secure a grant to “integrate helpful, privacy-respecting AI features” into GNOME… wouldn’t you? To be clear, this would include the features I mentioned earlier—not generative AI.
So I guess my question is (sorry this is so rambly, nuance is hard!!)… how would you as an ethical open source project with diverse interests and a community of users with strong feelings around “AI” approach this?
I almost feel like we need to distinguish the current generative AI hype from ML-powered features when it comes to public communication—but also lean into “AI” when looking at funding. And that distinction is… hard.
@cassidy This feels like the same conversation as analytics IMO. A lot of tech-y people are going to have a visceral reaction, but it can be implemented in a way that respects everyone involved and ultimately makes the software better.
@corbin maybe the solution is a separate org that just forges forward with both telemetry and ML, gets funding, and invests in making the whole stack better—/r/linux be damned. 😝
Even generative AI can be useful in open-source projects.
I'd love to see a GNOME API for image description (whether for the image viewer, the web browser, the mail client...).
This could be useful for the blind and visually impaired, but also for those who don't always know what to put as a useful image description on Mastodon.
Coming back to the question, for each GNOME application (core and circle) we should be able to list the features we'd dream of having.
It's a safe bet that a certain number of these will inevitably require AI, in one form or another.
Faced with the number of desired functionalities (which commercial platforms will sooner or later offer), we'll have to ask ourselves whether we really want to give up on them, at the risk of no longer being attractive.
@gnomelibre yeah this is where the terminology is even murkier—I didn’t consider image description as generative AI in my head, but sure, that seems genuinely useful and helpful if it can be done ethically and offline.
Gah, I hate that the terminology is so muddy. I was thinking of the ChatGPT/Dall-e/Gemini sort of iteration of the tech more when referring to generative AI.
@remusm there are grants and opportunities to work with organizations that could donate resources in the “AI space,” but nothing has been pursued to my knowledge. I’m just spiking out the idea and wondering if it would even make sense—and if so, how it would be approached.
@cassidy I see the predicament with needing to access available funding by using the terminology they expect, and there's little reason not to want quality machine learning based tools.
However I just want to float the concern of possibly of ending up with stale, silly looking nomenclature in otherwise quality software in the event the bubble bursts and "AI" terminology ends up with the air of NFTs and Web3 about it.
@cassidy IMHO even generative AI could be of help. It would help with translations a lot. I still contribute to Czech translations of GNOME and it's a really tiring and repetitive job.
In my experience DeepL translates 80% of UI strings just fine. If we trained some model on already translated gettext files, it would be even better and the role of translators would be reviews and fixing the <20%. It would help us make our software accessible to more users.
@sesivany@cassidy Machine translation has existed since at least the 1970s, has only marginally improved in the last ten years, and has tended to "hallucinate" more over the last 3 or 4 years - in my opinion as a professional translator, nothing has happened recently to make machine translation, even trained on specialist data sets, that much better or much more reliable and trustworthy than the past 10 or so years, with the only appreciable thing having changed being the hype around it. 🤷♂️
@sesivany@cassidy For me, MT review or translation takes about the same amount of time to achieve the same quality. The caveat being that the bar of quality is usually set lower for MTPE (Machine Translation Post-Editing).
@AdamBishop@sesivany@cassidy I agree, you have to really go through those "perfect" MT sentences with a fine-toothed comb because a lot of these tools seem designed to provide a slick surface which often hides serious mistakes. I've also noted that several MT user interfaces (especially the webtools) seem almost designed to encourage the translator to just breeze through segments, clicky-clicky, nice and quick, except no because it turns out it's stuffed with errors. I used to charge different rates for "light" and "heavy" post-editing but now I only have a heavy rate, because these jobs virtually always end up requiring a heavy amount of work!
@AdamBishop I'm not a professional, but I observe a major difference between DeepL/ChatGPT and whatever was available before. At least in Czech. I still translate manually, but when I'm out of ideas how to formulate a string in Czech, I put it in DeepL and get a good result in most cases.
@AdamBishop@cassidy I believe that dutiful work of a professional still produces better results, but it's not what we can afford, at least not for small languages such as Czech. We only have a handful of amateur translators dealing with hundreds of strings at nights. There isn't much time for quality like 15 years ago.
@sesivany@cassidy Sure, I understand. When resources are thin, you don't have the luxury of having the highest bar of quality, if you want to finish in a certain amount of time.
@sesivany@cassidy Though I think oftentimes the slow iterative, crowdsourced process is the right & ethical way for open source. People volunteer because they care about the project, & if they don't care they don't volunteer. I still think quality is is important, especially for minor languages, where the bar tends to be set low, and can even be very problematic for AI training and LLMs:
@cassidy what I don't see is why we need to seek funding for this in particular. If it's ethical, privacy preserving, and can be accomplished by Gnome hackers, why wouldn't we follow the normal processes for developing features? Why do AI features have to be attached to VC or angel investor funding cycles except to exploit people?
@cassidy put another way, if you have capital to distribute and you see Gnome offer to implement ethical, privacy preserving AI in a way that doesn't help you surveil or otherwise exploit anybody, and in fact shows people an alternative way to use this tech without submitting to capital, why would you help them? VCs and angels don't want to fund AI, they want to exploit and subjugate, and they think AI (the worst kind) will help them do that.
@ryanprior the hype is industry-wide, including VCs of course but also including grant programs and government funding. As a non-profit, I don’t think VC investment is even compatible with GNOME. :P
Every tech company is in full-on hype mode around “AI” regardless of how they make their money, so it’s on everyone’s minds.
@ryanprior the “can be accomplished by GNOME hackers” bit is a big assumption: there are tons of smart people doing GNOME things but there are not enough people with enough hours in the day to work on everything we want—the idea would be to attract funding to help fund those people (and hire people who currently can only volunteer on the side) to work on helpful features; in order to build these features, we would naturally be improving the a11y stack, underlying libraries, toolkits, etc.
@cassidy funding isn't attracted to "AI" alone, especially if it's local-first/local-only. Funding is attracted by the ability to send data somewhere, doing some magic process remotely, and then send back the result—because this is all the Scooby Doo meme:
@ebassi sure, if you are looking for VC funding. Obviously that model is incompatible with a non-profit like the GNOME Foundation—but that hype spills over into grant programs and even traditional fundraising to an extent.
@cassidy I agree with this and this was also my reaction when the internet was broken by the statement of AI by Mozilla.
My example was "I want to be able to select text from a video frame".
What I think I needs more framing is the "funding" part, as in "what kind of funding" --there are very toxic funding sources that shall be avoided for the long term survival of the project.
@gabriel part of that question is already answered by the GNOME Foundation being a 501(c)(3) non-profit; but I was specifically thinking monetary grants, but it could also involve working with other orgs that have experience in ML if they could contribute development or other resources to support GNOME.
Add comment