You’ve still got to avoid prompting for questionable code in the first place, eg, splitting SQL statements on semicolons with an ad-hoc regex is going to fail in edge cases, but may be sufficient for a specific task.
Yes more than sufficient for an internal tool - we can assume good intentions of the users of the tool since people want for this to actually work and have no intention of hacking.
Except now it's a vector if anyone gets access to this internal tool.
I would be fine with this for one off scripts but absolutely can not consider anything less than full sql parsing or something equally robust if it is exposed over the network, even if only internally and behind authn and authz.
For this reason, I tend to ask LLMs additional questions like: "show me another way to do this" or specifically "how would someone with a higher need for security write this?"... knowing that I'm likely to get a more refined answer from different sources that have probably discussed deeper security implications around the same goals, for instance.
Right on. These days my llm-assisted workflow feels very similar to the 20% of my day that I used to devote to code review, just now it’s more like 60% of my day.
I’m finding it’s more effective (and pleasurable) to write using GitHub CoPilot and CMD-RIGHT (accept next word). I put a detailed doc comment above and write in tandem with copilot. I’ve written the structure and I review as I write jointly with the model.
This way I don’t need to review a block of code I didn’t write.
<aside>I had an experience yesterday where CoPilot correctly freed all the memory in correct order at the end of a rather complicated C algorithm, even where there was nested mallocs.</aside>
Yeap, I want a code-review bot that just says "this is very improbable; are you sure you didn't mean x instead?"
The old Coverity used to achieve similar results in a different way, spotting probable mistakes based on patterns its heuristics found in the rest of the same codebase.
