Another fun mistake the AI analyzer found:
One of the curl test servers (for SOCKS) had a help text output listing around ten command line options. One of the options it listed was never implemented and thus didn't work. The AI found out and reported.
Kind of cool.
Pablo Martínez
in reply to daniel:// stenberg:// • • •daniel:// stenberg://
in reply to Pablo Martínez • • •A new breed of analyzers
daniel.haxx.seBubu reshared this.
Pablo Martínez
in reply to daniel:// stenberg:// • • •devSJR
in reply to Pablo Martínez • • •Yes, he discussed slob, but in all the cases it was purely machine fabricated and not checked by a human expert. And I think that's what it is all about. If there's a human in the loop, it can be a handy tool. On a side note, the European AI Act also requires humans in the loop for the final check. And back in the days when there were large hopes for decision support systems, people also said they don't work without the humans in the loop.
@bagder
daniel:// stenberg://
in reply to devSJR • • •Bradalot “”
in reply to daniel:// stenberg:// • • •@devSJR @pablo_martan
Another example of powerful tools used productively in the hands of experts (which famously create chaos in other contexts).
It's why "Don't try this at home." is a thing.
Osma A 🇫🇮🇺🇦
in reply to daniel:// stenberg:// • • •@bagder
Grampa
in reply to daniel:// stenberg:// • • •Nina Kalinina
in reply to daniel:// stenberg:// • • •I've been thinking about your post a lot, especially after seeing such tools at my $dayjob. I'm biased due to their ethical impact, but even without it I consider them, on average, harmful. I write code, I make sure it works for my usage, I write tests, I do linters and static analysis, I do a peer review to share the knowledge and get external inputs. And then this thing, supposedly state of the art, goes over my code, mansplains it to me and finds either a false-positive (I wonder who removed false-positives from the lists you've got?), or a nit ("don't forget to add an extra check here!", "The comment is stale!"), or a misguided optimisation possibly introducing new bugs. I spend lots of time thinking over those useless blanket reports that ultimately don't matter because I have empirical evidence that my code works for my use case.
I have seen so called AI tooling generating "helpful reports", but they cannot replace decent tooling and tests. And yet some people replace their LSPs with LLMs :/
daniel:// stenberg://
in reply to Nina Kalinina • • •@nina_kali_nina all analyzer tools, including compilers, give a certain amount of false positives. I don't think we should expect AI tools to be any different. As long as frequency is manageable and there are decent ways to inhibit them.
The AI tools I've mentioned recently don't seem to have much more false positives than the state of the art static code analyzers we use also do.