Im working on my own coding agent soon to be released.
While you might think "yey another coding agent" i went down some very different paths than other open source coding agents do right now. Ill not bore you with a ton of detailed breakdowns and rather list the main points that are relevant to understand why/how i build it.
Premise:
-The agent is meant to run against selfhosted environments first like Ollama etc with a focus on non cloud sized models, it should work properly on something like a 27b model already
-The agent with full intention trades execution time vs reasoning and result quality
So what does it do different:
1. The agent, instead of trying to let the model solve a whole task in 3-5 inferences, rather breaks down the "thinking process" in smaller chunks, basically decomposing the task into smaller tasks. While this is not a completely new concept, the agent will break it down to really simple single step variants which even for a smaller model can properly be solved. This involves not just file editings but all points along the way from interpreting user message to planing a task to defining acceptance criteria etc.
2. The agent is build in a way that it, apart from some fixed steps such as detect user intent at the beginning and synthesize response at the end, it will by itself decide what is the next proper step to execute/do based on the context i provide. While doing so the agent also revalidates its current execution and if it spots that, given the information i collected since it planned and started running, the current plan is not detailed enough or is missing out due to something he didnt "know" when he was planing, the agent will dynamicly mutate the coming plan stats, as in restructure/decompose/etc in order to have a properly layed out route to fulfill the task.
3. The agent has a rather complex system of how his context is composed which is combined by things like the chat history, previous step results, a backlog of what "single steps" it has done (updated file x , read range from file y etc) and in that it is strongly reason driven, so it will for most of the things it does also provide a small reason explaination for why it did that. Than the agent also has a working memory that contains facts and decisions it made along execution. There is more but thats just some thingsd to mention. The whole memory system and context is rather complex (tho not complicated). When i release it ill properly break down how all this works in the docs.
4. The agent provides full observability in the ui. At every point of execution you can basically in detail see whats in memory, whats the current composed context, whats the plan and what plan steps depend on what previous steps, you can look at artefacts it build and also you have basically a complete audit log of all single "actions" it has executed and can look into their details.
5. The agent allows to while in an execution send steering messages, meaning the message will be, using relevant current contextual information about its execution, send as inference with a prompt askind the agent to interpret on if its relevant, and how does it impact the current execution/plan. It than can decide either just do alter what it currently does, or even do a plan mutation in order to accomodate the users steering information.
6. The agent within its capabilities has a lot of error correction/self reflection logic. From dynamicly fixing json including a fallback to let LLM fix a response json, upon to if executions fail to much it will itself do a hypothesis on why this is happening, formulate a critique and with that reevaluate if it wants to try the steps again with the additional information, or if it wants to mutate the plan to accomodate the just spotted problems.
Theres quite some more i could list but i guess thats enaugh for now. The big trade off as mentioned earlier is execution time. In comparsion to cloud native agents, which will do giant editings and reasonings in just a hand full of inferences, my agent uses a ton of inferences. The big difference is that this allows (a) the agent to more precisly focus on specific tasks rather than overloading it with to much at once and (b) alot better spot problems/mistakes itself and adjust its execution plan to accomodate those without the user having to reprompt the agent 10 times until something is actually correctly solved.
Apart from the pure coding capability, the very same actions that allow for higher quality reasoning and coding at the same time allow the agent also be a good analysis and discussion partner.
So : what is it than used for if inferences and executions can take very long? Well for me its in terms of coding capabilities a hands off agent. Its meant to be informed once at the beginning with relevant details, and than should be fully capable (as long its in the capabilities for sure) to fulfill the task without the user having to permanently keep an eye open. I want to free time for myself and not change my time from coding to prompting.
Ill wrap it up here and say : the Agents name is "Loa" and i probably will post it on hackernews as soon i got a relatively stable beta to release. Im close to being fine with a beta release but i want to test some more runs before i publish.
Tbh in a time where more and more applications are shipped as vibecoded electron based bloated bug-collections that are slow and take stupid amounts of resources i'd rather have even more TUI's than less.....
Funny, i did this myself some months ago - its actually not that tricky to run a llm in your browser. Tho, the problem is that the models fitting in there are well relativly "tiny" in my opinion.
I have started working on a 1 Person RTS project (crazy right? i know :D)
Basically a 2d rts, fully orthogonal view, plays in space with spaceship combat.
You start with a deployable base, kinda like the old C&C, which you can place around a minable planet. Than from this you can place further structures arround that planet including the typicals, mining (2 types) of resources, researching stuff, and building ships.
The 2 things i focus the most on in this game on are
1. Responsiveness. Im a strong fan of starcraft 2 (and brood war) and have spend tons of time playing sc2. I love this super responsive controles and its the only thing i will settle for in terms of controles
2. I love the game starsector (yes no rts) for the way its ships are designed. You have a base hull that has some stats like hp / speed / etc - than this thing has basically mountpoints where you can fit weapons into. I always loved this system. So what i build is a game where you can (outside of battle) define your desired ship blueprints combined of a hull, weapons and "inbuilds" (as in upgrades). You can than just build those ships ingame.
So ye made quite progress so far , not sure tho what will be the first mode i focus on once i get a playable (as in demo) state.
Btw using godot engine as base, but some logic is running as c++ gd extensions (rts is heavy calcs , gdscript is just not fast enaugh (or im just not a gamdev ¯\_(ツ)_/¯)
The following rant is not against the owner/project - but...
What an irony. I cant publish a attack surface mapping / pentesting tool i wrote which runs fully deterministic and really controlable due to "dual use" legal problems - but llm driven tools hit public space......
This is really regrettable. I think the biggest issue is that once such laws or case law are in place, it is really really impossible to repeal them, or to replace them with something more modern. So we're still applying horse laws to cars 100 years later.
No im referring to the legal terms of germany, the country im residing at. Our laws regarding "hacking" are arguable the strictest and worst.
The problem is that they are formulated in a way that it is super easy to have your software being possible "dual use" and that a judge has to decide if its fine or not. Making it worse it also states your "intention" which well is impossible to proof - if the judge says he doesn't believe your intentions are only good, you can literally get massively sued.
So ye i could move to another country and than publish it - apart from that i can let it rot on my hdd (which is prolly what will happen).
Edit: Additionally mentioned, it is not just the publishing in germany, even the facilitating already which is why i don't even have an article about it (any more).
So would it be fair to say this is less about deterministic vs. LLM-driven tools, and more about software published out of New York vs. software published out of Germany?
Yes and no. I have seen (not wanne name them now to not bring them into a bad context) several tools coming from germany which are well in questionable contexts but due to basically beein advertised as full LLM/AI driven its somehow not on anyones radar. So that is kinda a thing.
But sure location itself also plays a role no question.
Intentions are very often a factor when it comes to law, and rightfully so. The problems come when you classify some category of software as "dangerous" by default. Software is special in that it can't be a "controlled substance". So everyone needs to have the tools to increase their security. That includes access to "offensive tools" to study what they do. A recent example was the Huggingface attack.
As I understand the hole shebang, the German law is about "does the court think you are `hacking` something?" -- this is against the law.
Yes, there was one German pentester that got sued, because he reported a BASE64 encoded, hardcoded authentication token in an application.
Not that I would wish him anything like that and am ashamed by the outcome, I also recall that he might have tried to put pressure on the vendor by doing an interview with a blogger or so?
On the other hand, I do not know of any cases regarding publication of dual-use tools as OSS.
If getting it out there in the world is more important to you than stamping your name on it, pass it out to a friend who can publish it. Continue any related development in private. They can't stop you from distributing something they don't know you're developing.
Well its not like i didn't consider it, but it heavily relies on stuff that i have written and that is well only used by me till this day. So arguably, publishing it this way it would still have like "points" which make it extremely related to only me.
Also asked the lawyer in consulted about it... i would be a heavy gamble
I'm asking cuz I started devloping a c2+agent+BOF kind of thing with custom bytecode vm for the lulz (to learn how stuff works nowadays) and it's on tangled and github :/
Yes. More than one. This one was especially "interesting", a security researcher was tasked by a company to evaluate the ERP (I believe would be the acronym) software. Diiscovered an external database connection, looked at it, discovered this external DB contained sensitive information from other clients of that vendor - reported it, got sued, lost.
Ouch, that's pathetic. Hm, maybe I'll leave it then. Not worth the hassle. Although I don't really think someone would hunt me for random non-prod github repos
Ye you should be really careful on that one. If just somebody with a bad mood reports you, even if you have absolutely good intentions, it can cost you and if its just in spendings in a lawyer to proof that your fine....
Completely understand, the legal landscape has really shifted around AI/LLM tools. I see tools drop everyday that spit in the face of DMCA/Copyright law but they skirt by mainly because they leverage AI
That has nothing to do with LLMs or not. If they were to "code" the same tool with an LLM, or build a LLM version of it, in Germany (where rthey are residing) they would face the same consequences.
Our legal system in that regard is total BS, when it comes to the so called "Hacker Paragraph". This comes with a sentence tht can range from a fine to up to one year prison time, just for producing such a tool. So if the police would actually - even if they had not published it - computer... well. They could be potentially "f**ed" (depending on how a judge would decide).
One side i really like it - i also love to play around with funny ideas - but have to say if i would read more than like 2 sentences with that font i'd throw up xD
Some people say im rather pessimistic - but i still kinda hoped that the EU will not go down this path.
That said - i was obviously wrong.
I don't really know what else to say than : this breaks a fundamental foundation of privacy. Sure i was never 100% aligned with EU level decisions (neither with my residing country one's) but that's just... wrong on so many levels... how can i even still argue to people that EU and democracy are good if this is what its heading for?
Usually i try to explain stuff with common grounds, the bigger picture or whatever is fitting. You can't have everyone happy on every decision.
But at this point im lost of arguments. They just betrayed every citizen from my pov.
So whats my point of this comment? Maybe i need to vent in an environment where i think people might get how bad this actually is, since people IRL around me don't seem to remotly understand the impact....
Really you should have felt betrayed back in 2011.
This isn't Chat Control 2.0 - which is the thing that has been getting attention for the last few years.
E2E encryption should not be broken, we should find other ways to catch and find predators.
Scanning messages in platforms (Chat Control 1.0)? I honestly thought this has always been done, I don't consider any platform private.
A lot of Internet regulations are just catching up to ones that exist in meat space.
I think what scares people when the regulations come to the Internet is that they are highly automated, and I think there's potential for mass injustices when automated systems.
We’re talking about erosion of privacy and draconian internet surveillance. Brexit was a major screw up and I’m not here to defend any side, just to dispel this illusion that somehow the left are the good ones and the right is the bad far right.
While you might think "yey another coding agent" i went down some very different paths than other open source coding agents do right now. Ill not bore you with a ton of detailed breakdowns and rather list the main points that are relevant to understand why/how i build it.
Premise:
-The agent is meant to run against selfhosted environments first like Ollama etc with a focus on non cloud sized models, it should work properly on something like a 27b model already
-The agent with full intention trades execution time vs reasoning and result quality
So what does it do different:
1. The agent, instead of trying to let the model solve a whole task in 3-5 inferences, rather breaks down the "thinking process" in smaller chunks, basically decomposing the task into smaller tasks. While this is not a completely new concept, the agent will break it down to really simple single step variants which even for a smaller model can properly be solved. This involves not just file editings but all points along the way from interpreting user message to planing a task to defining acceptance criteria etc.
2. The agent is build in a way that it, apart from some fixed steps such as detect user intent at the beginning and synthesize response at the end, it will by itself decide what is the next proper step to execute/do based on the context i provide. While doing so the agent also revalidates its current execution and if it spots that, given the information i collected since it planned and started running, the current plan is not detailed enough or is missing out due to something he didnt "know" when he was planing, the agent will dynamicly mutate the coming plan stats, as in restructure/decompose/etc in order to have a properly layed out route to fulfill the task.
3. The agent has a rather complex system of how his context is composed which is combined by things like the chat history, previous step results, a backlog of what "single steps" it has done (updated file x , read range from file y etc) and in that it is strongly reason driven, so it will for most of the things it does also provide a small reason explaination for why it did that. Than the agent also has a working memory that contains facts and decisions it made along execution. There is more but thats just some thingsd to mention. The whole memory system and context is rather complex (tho not complicated). When i release it ill properly break down how all this works in the docs.
4. The agent provides full observability in the ui. At every point of execution you can basically in detail see whats in memory, whats the current composed context, whats the plan and what plan steps depend on what previous steps, you can look at artefacts it build and also you have basically a complete audit log of all single "actions" it has executed and can look into their details.
5. The agent allows to while in an execution send steering messages, meaning the message will be, using relevant current contextual information about its execution, send as inference with a prompt askind the agent to interpret on if its relevant, and how does it impact the current execution/plan. It than can decide either just do alter what it currently does, or even do a plan mutation in order to accomodate the users steering information.
6. The agent within its capabilities has a lot of error correction/self reflection logic. From dynamicly fixing json including a fallback to let LLM fix a response json, upon to if executions fail to much it will itself do a hypothesis on why this is happening, formulate a critique and with that reevaluate if it wants to try the steps again with the additional information, or if it wants to mutate the plan to accomodate the just spotted problems.
Theres quite some more i could list but i guess thats enaugh for now. The big trade off as mentioned earlier is execution time. In comparsion to cloud native agents, which will do giant editings and reasonings in just a hand full of inferences, my agent uses a ton of inferences. The big difference is that this allows (a) the agent to more precisly focus on specific tasks rather than overloading it with to much at once and (b) alot better spot problems/mistakes itself and adjust its execution plan to accomodate those without the user having to reprompt the agent 10 times until something is actually correctly solved.
Apart from the pure coding capability, the very same actions that allow for higher quality reasoning and coding at the same time allow the agent also be a good analysis and discussion partner.
So : what is it than used for if inferences and executions can take very long? Well for me its in terms of coding capabilities a hands off agent. Its meant to be informed once at the beginning with relevant details, and than should be fully capable (as long its in the capabilities for sure) to fulfill the task without the user having to permanently keep an eye open. I want to free time for myself and not change my time from coding to prompting.
Ill wrap it up here and say : the Agents name is "Loa" and i probably will post it on hackernews as soon i got a relatively stable beta to release. Im close to being fine with a beta release but i want to test some more runs before i publish.
reply