@jasongorman I have worked with developers who think exactly like that 😂
This is the reason why I currently actively discourage the use of GAI when writing code. It’s snake oil. There might be potential there, but not for another 10-50 years of heavy investment and development. I can’t help thinking Step 1 has to be generating a training set of data from people who know what they are doing rather than simply scraping the nonsense on the internet.
@thirstybear That's a house of cards, though. ML researchers keep telling me that you have to sacrifice the statistical fluency of the model if you use a significantly smaller data set.
But what the researchers seem to be saying here is that AI reflects the current wider understanding of software development as reflected in the training data. Which IMO isn’t great. “Roll up! Roll up! Buy our snake oil that lets you generate average code 10 times faster!”. That’s right up there with “Offshore your development, and fail to deliver 10 times cheaper!”
So maybe simply improve the quality of the training data to shift the average quality?
@jasongorman This is why I say that anyone using “tools” like Copilot need to have sufficient industry experience and critical thinking skills to pick the diamonds from the tsunami of BS it spews. The majority of time it’s quicker to bypass it. Think “net-negative developer”.
@matthewskelton I'm running a few experiments to try to understand how some of the solutions to my Code Craft Foundation individual exercise came about. I saw a lot of "A.I." code generation in their screen recordings.
These LLM tools have been trained on a lot of student exercises and examples from books. How well they solve exercises seems to me very poorly predictive of how they behave in realistic corporate codebases. Is the student simply learning a tool that will stop working when they need it for real?
@emilybache@jasongorman@matthewskelton I've been trialing Github CoPilot, while working on a mature (15 years) commercial code base, and it's almost universally wrong with it's suggestions, such that it doesn't save any real time once you've corrected it.
It has a bad habit of forgetting the context of the code your working on after a few steps, and inserting references to properties and methods that don't exist.
Where it has been valuable, is that I'm working in an unfamiliar language, and it has been quite useful in surfacing the language conventions, keywords etc, even if they are presented in solutions that are wrong for the scenario.
@jasongorman@nonspecialist@aj@emilybache@matthewskelton I'm wrapping up a 3-week training program for a small group (5). At the outset, I told them the state of AI assistance today is such that an experienced developer can benefit from it because they have the background to assess the suggestions it makes. I advised them not to use AI in the course because it can mislead novices, who can't judge the quality of the suggestions. They worked without AI and I think they learned well in that way.
@jasongorman@nonspecialist@aj@emilybache@matthewskelton But that would be a good way to learn how to use a translation app. It's like learning to ride a bicycle with training wheels. One becomes an expert with training wheels.
Add comment