Good in terms of prompt communication and fix. Absurdly bad in terms of reward.
Earlier in the article, it mentions that Baseten is valued at $13B. They can't dig into their couch cushions to give a few thousand dollars to the researcher privately disclosing a bug that let an attacker escalate to admin in their GitHub org?
This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
This is two companies working together. Most of the comments below are assuming this was an independent security researcher doing work on their own time. This was professionals doing work for their companies on both sides.
> This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
The reason they were looking for bugs was in the context of a B2B relationship, not as a someone independent on their nights and weekends.
If they give them any additional compensation it would probably be in some amount of free or discounted services, which is what they’d want anyway.
Swag packages like these are a token of appreciation not a reward.
The front page post in HN here is worth far more than few thousand dollars , don’t think either organization is operating under purely financial transactional nature .
Most people who find a dropped wallet will return it without evaluating the market value of your compromised identity or the contents of the wallet .
Grateful owners may buy you a beer that doesn’t make them cheap , not everything is evaluated in purely money terms, and that is a good thing ?
> The front page post in HN here is worth far more than few thousand dollars , don’t think either organization is operating under purely financial transactional nature .
not always, especially if its just someone independent. iirc there was a guy here not too long ago who started dropping Windows 0days because Microsoft couldn't be assed to process his bug reports
That actually supports the point that people aren't acting under purely financial motivations. If the guy was purely following financial motivations, surely he would have chosen to sell the vulnerabilities to the shadier side of things. Instead, he dumped them publicly, burning their value while amplifying the "fuck you" factor to Microsoft.
Ignoring reports, or just fixing the vulnerability without acknowledging the work put in by a researcher, is rude and invites rudeness in return.
Microsoft runs a bug bounty program. NightmareEclipse (that’s the researcher’s handle) allegedly participated and Microsoft did not honor their part of the bug bounty program terms.
This is a completely different situation - a company evaluates the security of a prospective vendor prior to entering a business agreement.
> iirc there was a guy here not too long ago who started dropping Windows 0days because Microsoft couldn't be assed to process his bug reports
Did that ever actually happen? I remember him threatening to start dropping 0days and getting a lot of press coverage for it. When I tried to look it up I didn’t find anything at the time.
“Nightmare Eclipse released these zero-day exploits as part of an ongoing dispute with Microsoft over the company's bug bounty and vulnerability disclosure practices. […] Since April, the anonymous security researcher has disclosed a long list of zero-day flaws, including ShieldBreak, LegacyHive, RoguePlanet, BlueHammer, RedSun, YellowKey, GreenPlasma, MiniPlasma, and UnDefend, targeting Microsoft Defender, BitLocker, and other Windows components.”
Wallets usually belong to real people with lives. We can empathize with them. Companies are not people. And they also don't and can't empathize with you.
Companies are 100% people. The fact that companies, their CEOs, and employees are not treated like people is exactly reason why humanity is in the shitshow show it is right now.
>Wallets usually belong to real people with lives.
So does data ? it belongs to real people.
I would imagine baseten's customers and eventually their end-users[1] were also grateful that their data was not compromised here and the disclosure was responsible.
[1] There is a pretty good chance you and I could be using services who are using baseten
"and can't empathize with you" - I don't really understand why such a perception of companies has been regurgitated and reinforced so much in US public, to the point where it's a blank excuse from ever expecting such a thing from a company. It's not true that it can't. The only reason to keep repeating that kind of worldview is to absolve companies behaving in shitty, toxic or downright evil ways.
The law doesn't say companies MUST choose the most profitable choice at every turn, and even explicitly allows for good treatment of customers, community, employees etc as a viable business strategy (even if it's sad that it must be justified in that way).
I agree that this notion that companies make these decisions is bad, bad on the grounds that it's people working for those companies that make the decisions, they hide themselves away, but the company itself isn't doing anything - is always a human making the decision
I think the point is that companies are purely legal entities, and as such, cannot feel, much less empathize, simply by virtue of them not being living things
That's nonsense. "Companies" are not something non-human, they are run by humans, who do feel, empathize, and are living beings. Without these living-being humans, there would simply be no "company".
Now how those humans that run the company behave is another thing - they are free to be greedy assholes, and a lot of them are, and some of them aren't - but that's still a human thing.
Yes, they are not humans. They are not even living creatures. They are mostly-legal entities mostly for the purpose of contracting with humans or other legal entities.
> they are run by humans, who do feel, empathize, and are living beings
This is usually true, but it is orthogonal to whether the company (a legal entity) itself is a biologically living creature, which is the only thing capable of feeling*. To put a point on it: my lawnmower is also run by humans, but it does not have empathy for any grass or people that gets in its way. It mostly just goes where it is steered. A company is like that, except with less touching grass.
* — unless you want to argue semantics about what "feeling" means, even though the discussion is about "feeling" and "empathy" in the way humans experience it, and how that form of "empathy" does not exist for a nonliving legal entity which may or may not employ any actual humans
I think the only misleading part of this situation is your naive and self-centered definition of "trust", and the assumption that so many others think similarly enough that they need to be warned.
I trust a business to fulfill their obligations as stated in writing for the money paid. I do not trust them in any other way. Nobody should "trust" or depend on undefined behavior. Common sense can only ever be as common as you expect.
> expecting compassionate or empathetic behaviour from companies, and [another thing, which is not the same thing as the first thing but merely similar or related]
First, you're being petty and just fighting fire with fire. Second, most of this research is fairly trivial.
What you're instead encouraging is a race to the bottom. You're not going to kill off the companies you hate by withholding information. You don't even have that power anyway because by its very nature, security research is not secret. You're really just encouraging pessimistic groupthink and bad faith. This is why businesses can't be more open about their flaws. It's not that they're stupid and incompetent, but that the pitchforks come out. These are the seeds of dystopia.
They would have eventually figured it out, but as an unfortunate incident with an outsized effect. As much as you wish it to be true, even the worst of these incidents will not kill their business. As much as you hate these businesses, their financial momentum will eventually cause the public to depend on them more. There's more at stake here than anyone's personal gain. It's naive to think otherwise.
You're just manifesting broken windows and ignoring litter thinking you're fighting the man. This is straight up ghetto punk ass behavior wearing a white collar.
> I'm saying serve yourself, not them. if you have say, a 0 day on your hands, do what serves you best. is that "ghetto punk ass behavior"?
Yes.
If you have say, managed to find an overlooked passage into an ostensibly high security building, "doing what serves you best" such as selling the information to some thugs, is in fact that kind of behavior.
for real security bugs, like, you can literally sell them to brokers who sell them to governments. would selling stuff to the CIA be ghetto?
morally, it depends. but after seeing so many posts of e.g. Google cheapskating on bug reports, it really makes no sense to me to participate in such a broken system.
this case however is quite different as it was a B2B encounter and during vendor vetting
like to me it just seems like a fair deal, if Google wants their bugs patched (which they can definitely afford to do) they'd just pay properly for serious bugs and so on, and everybody would be happy. it's not some kind of thing where they can't do anything about.
maybe you can understand the angle I'm coming from?
This was a potential customer reporting a result of an audit of a tool they are evaluating. This is frequent and normal activity in enterprise deals. Most of the time such reports are not critical vulnerabilities it would things like tenant configuration -what business would like versus what CISO will accept or risk acceptance of the product they are buying with monitoring or other prescription on access restrictions or a DPA and so on.
It would be novel business model to spend ton of money in getting a prospect to late-deal stage where they are ready to do a security audio for you just so that part is "free" .
Most companies wouldn't disclose(to the public) even if it was serious , that is not their job, they will report to internal teams and re-review on fix. Strix.ai has a benefit in doing so as they sell a scanning tool for this purpose so we get to hear of this.
Yeah companies need to quickly understand that having good actors try and hack you is a good thing - those hacks get reported and another door gets sealed shut for bad actors.
This is more true today than ever before as the bar for a successful attack has never been lower. We’ll see a resurgence of the script-kiddie, or shall I say, vibe-kiddie :-/
The researcher in this case was doing a security review for their company who was a potential customer. Sending potential customers more than a token amount of cash is usually prohibited by corporate ethics rules for obvious reasons.
That's incorrect. It's not only perfectly acceptable, but absolutely vital, to pay someone for their services (incl a customer) for assisting with an existential threat against the corporation.
Any counsel or HR who would draft a corporate ethics rule that wouldn't allow for a bug bounty to be paid out on a massive vulnerability, merely because the person was "a potential customer", should be immediately replaced.
if it were my company I'd not pay a dime if the researcher was going to make a big public blog post about a security issue in my infrastructure that I promised customers was secure.
I'm sure the cash value of the advertisement here is worth more than a bug bounty would pay.
> This sends the message that honest researchers should not waste their time looking for vulnerabilities in Baseten, but it's a good target for criminals who want to monetize these vulnerabilities.
of course, these companies want you to sell vulns to brokers and other orgs. they don't care about bug reports.
I answered this in another comment,[0] and I don't think there's widespread agreement on this, but I think design docs should be a short-term doc that lives until the design implementation is complete. I don't think design docs are the right format for a document that has to evolve alongside the code forever.
Fun piece of trivia, Joel published one of his functional specs.[0]
As a huge fan of Joel's writing and engineering ideas, I was actually underwhelmed by his spec. It wasn't bad but it also felt like he missed a lot of opportunities to articulate design decisions to the reader more quickly or clearly.
One obvious mistake is that there's over a page (in a 20-page spec) just dedicated to coding conventions and what prefixes variable names will have. I think Joel later conceded that it was a mistake to cover naming conventions in a spec, though I can't find a link now.
I definitely agree that it's not perfect. I think he's a great starting point though, because when trying to get engineers to document stuff (something typically approached with similar enthusiasm to having their teeth removed with a hammer) it really helps if the "how to write a spec" doc is somewhat fun to read.
Unrelated - I really dig your "my [x]th year as a bootstraped founder" series.
My rule of thumb is to ask myself, "Is there a chance my reviewers would not have signed off had this been in the design doc they reviewed?" If the answer is yes, I send it out for a follow-up and explain why I had to change the design.
In my experience, the response from my reviewers is generally, "Yeah, that's fine." It's a combination of (1) the practical limitations that it's hard for them to get the whole design back into mental context to argue about it and (2) they trust that I'm taking the design seriously and have thought this through. I think occasionally, I've sent a post-approval change out and someone points out something
It's common to encounter a curveball nobody anticipated at design time, but if you just go rogue and unilaterally make design decisions, it degrades trust and undermines the review process, so I want my reviewers to know that I'm taking their feedback seriously.
> Isn't much of this made redundant by being part of an existing system?
I haven't found that to be true in my work. If you're only making a minor change to an existing system, then you may not need a design doc, but a significant change to an existing system has as much, if not more, complexity and ambiguity than greenfield development.
> Also, this level of detail is a recipe for being outdated once the issues, compromises and compromises starts coming in
I think this is what people typically get wrong about design docs.
I don't think design docs are a good medium for being the perpetual, living description of the system. I think design docs should capture the design at the time of implementation. You should modify the design docs while you implement the work called for in the design document, but once you're done with that work, you freeze the document and preserve it for posterity only.
The design doc is about a specific change to the system. If you need a doc to describe the high-level architecture of the system as it evolves, that should be a different doc.
I'll admit a lot of bias because I think design docs are extremely useful, but I find that when people hate design docs, it's almost always for one of two reasons:
1. The developer has worked on teams where design docs are viewed as a pointless ritual, so authors treat them as a pointless requirement and write bad docs and their teammates view them as pointless so they don't bother giving useful feedback, reinforcing everyone's belief that they're a pointless ritual.
2. The developer does not like other people questioning their engineering choices, and they know that it's harder for their teammates to push back on finished code than a design doc. Investing in the implementation before design changes the calculus to bias in favor of whatever's already implemented rather than what would have been the ideal implementation. Plus, it's harder for the team to review design decisions of 10k LOC than a 5-page design doc.
I think your framing is fair here. But I'd like to offer an even more complicated/nuanced take:
Designing in a group can be very difficult, and doing it well is a skill set that most people don't naturally have.
I think this explains your point 1. Why do people view designed docs as pointless? Because they really don't have a vision or model for what and effective and healthy collaborative design process would look like.
> Designing in a group can be very difficult, and doing it well is a skill set that most people don't naturally have
Not only is it a skill every participant needs to have, they also all need to have a similar amount of competence and knowledge about the domain as well as the current implementation, otherwise it's mostly pointless ime.
But if all ven diagram circles overlap ... It is nice.
I can count the times this materialized (in my professional life) on one hand. So I'm generally more towards the "make a prototype, then explain it to the others. Either it's the baseline for the discussion or the illuminating event that clears up wherever this approach works with that team.
The design "doc" is needed, but not the static doc for printers, a more dynamic one where you don't need a 50 pages doc with lots of links between pages, but a very good dynamic diagram with some text.
1b: the docs are a pointless ritual that sometimes ends up taking multiple times longer than implementing, and if too many people see it they start asking things like "what are your KPIs" and "when are your deliverables synergized" and "have you written the oncall runbooks yet? what's the protocol spec for that?" and "what's your projected uplift". if it's less than 5 pages of text, you're dinged on perf because your documents aren't detailed enough. for projects like "we should add a small in-memory cache to this slow area".
so you're best off writing a small one that you do your best to hide, and make a few fancy ones per season with graphs and absurd will-never-be-implemented details for the higher-ups to be distracted by.
YES - and 3 the the assumption that it needs to be a certain length before it can be considered a design document.
Esp. in the age of AI a little guidance to brainstorm on before a random project prompt goes a long way. (Not saying people that don't use design docs just code away thoughtlessly) - experience goes a long way too that's why there's success stories with and without design docs.
So you need to believe in design documents, and if you don't like them you're probably trying to manipulate people? But have you justified design documents?
I can think of situations where design document(s) would be a clear use, and I think that would be a better way to respond.
Num. 2 especially relatable.
A good mark of high quality professional is if he presents his plan before execution to hear feedback and comments - even if they are totally against his original idea, and he can then take this feedback and incorporate effectively in a re-design.
If they don't have the required knowledge before coding, how are they going to have the required knowledge if you skip the design and just hand them the finished code?
3. The business cannot figure out a direction so the developer can either churn on design docs fruitlessly or make prototypes that visually show the business people what our options are in order for them to make up their minds
Not OP, but I think they're way more essential with AI doing a lot of the coding. The biggest thing that AI, even the frontier models, is not great at is staying on topic and actually finishing a project with reasonable priorities instead of ratholing on insignificant details or claiming it's "finished" when it's half done.
The most important thing that a good design doc does is specify what's in and out of scope. The second most important thing is to precisely define common vocabulary - what are the important concepts in the problem you're solving, and how should they relate to each other? All of that information serves to ground the day-to-day work in what's important. I find myself starting every Claude session with "read this doc and get familiar with the world, then we'll get to work on a part of it".
(The same is true when working with humans, especially but not limited to junior engineers who aren't used to managing a project longer than a week or two. AI coding agents just never grow out of that phase.)
For an entire project? Yes I agree. For a feature or a submodule? I think when you work with claude to develop a plan, it's generally pretty good.
I guess my question stems from being rigid how a design doc should be defined, argued over, and then executed by humans. I think some of the details simply don't matter, and if they do, they often can be changed relatively quickly in order to adhere to the new requirement.
In the age of AI coding, code is cheap. Getting the requirements and high-level architecture nailed down is where the hard engineering challenges remain.
We have debated this a lot in our organization. We are tired of seeing low effort Tech docs that puts the onus on the reader than the writer. I think that the writer should spend at least an order of magnitude of time more than the reader. If not, then the design doc can just be the LLM prompt that generated the document.
I have actually resorted back to hand crafting TDDs and focusing on 1-2 page docs. It is a great way to organize my thoughts and create a shared mind space among other engineers. My 2 cents.
I agree that we should basically be requiring hand-written-only design docs, because it should force people to make sure they know what they're getting someone else to read. But there's two problems I run into:
1) A lot of people who write design docs, RFCs, etc, don't write them well. I end up needing to get them on a call and explain their entire idea to me because it's the only way to pull the details out of them.
2) Regardless of how much I write by hand, I still have engineers who are so incredibly lazy that they just don't read the docs at all. They can't be arsed. So I have to get on a call and basically explain the whole doc to them.
This is starting to lead me back to what other people hate: meat puppeting. Telling Claude my idea, Claude writes it up, and I ask that engineer to ask their Claude to read my Claude output and summarize it for them. I really want a better solution, but our engineering management is almost nonexistent, so nobody does anything they don't feel like doing.
IME when starting a project from scratch, detailed upfront architecture specs are pretty much required to keep LLMs from flailing around too much (unless of course you build another cookie cutter CRUD webpage, those can simply copy paste from the millions of examples on the internet).
In a way it's a return to waterfall, just with faster implementation phases.
That means you could choose to try three (or more) genuine implementations and explore their tradeoffs, instead of making three proposals in a document with one recommended (and the other two usually only provided for contrast).
I do think the design is important to keep around - in particular, the constraints, the communication points, schema, tacit things that might not be clear in code. I am not certain that the design should precede the implementation for features below a certain size though.
Larger efforts need milestones and collaboration and will have multiple people doing implementation, so there's more need to agree schemas, APIs etc up front there.
Agree, but in my experience that doesn't change much about the design doc.
I think it's helpful to the author to be able to say to an AI agent, "Hey, put together this quick prototype," and that informs the design doc. But if the goal is to review the design decisions with the team, I don't see how you get around the design doc. I don't want a teammate to send me 10 KLOC of AI-generated code and ask me to review the design. Even if you told AI to try 10 different ideas and pick the best, I don't trust AI to make the same decisions as my human teammates.
I'm not suggesting using AI generated code as a proposed design.
I would try and get an understanding of design space by giving a good agent a high level goal and seeing what it does, then getting a summary of the approach.
When you do this several times, especially if you give it a steer on some non-functional requirement, you can compare and contrast different approaches.
The idea isn't to prototype so much as to gather information by doing. Prototype, to my mind, suggests other things; shortcuts, stubs, incompleteness. I would actually ask agents to do the whole thing, and find out the full scope. It can be particularly useful revealing side effects.
Pair it with code auditors wearing different hats, of course.
I find that LLMs are still worse than humans at limiting complexity, which is one of the most important outcomes of a design review.
If I tell a senior SWE that I'm creating a Discourse-like discussion forum, and I want users to have three options for selecting an avatar: (1) import from Gravatar, (2) upload a JPG or SVG or PNG or GIF, or (3) let the user draw their avatar on a canvas, the LLM will happily go and design that and write a 5 KLOC implementation, whereas a good SWE would push back and say, "That's like 10x the complexity of just allowing JPGs. How about we simplify it to say that in v1, the only option is to upload a JPG."
I've tried working with Fable/Sol and saying, "Look for features that we can simplify to reduce complexity," and they don't understand. They'll guess at features we can cut entirely, but they fail to see how to capture the essence of the feature without the complexity.
I've noticed this a lot with Fable recently. Like I'll say, "Show an error message in the web UI if X fails," and Fable comes back with this like 800 LOC error message generator that has switch-cases and combines inputs from three different sources when all I wanted was something like, "Update failed: database is locked."
I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy? If it is the secondary outcomes that are the enemy, and AI actually effectively allows you to mitigate those outcomes (debatable! I debate this with myself all the time!), then maybe we should embrace the complexity (or the agent should on our behalf)
> I do agree with you put I would push a little further - is it that complexity itself is the enemy? Or is it that the secondary outcomes of complexity (bugs, more effort to make changes, confusing code) are the enemy?
I agree, but I think we're still a long way away from being able to trust AI to manage all software complexity for us. For one, LLMs frequently get tripped up by their own complexity. But even if the complexity didn't make LLMs more error prone or expensive to run, you still often need a human in the loop to understand what the system does.
I think of it kind of like compilers. Compilers do a good enough job that 99% of developers don't understand code at the bytecode or machine instruction level, but if we lost that last 1% of programmers who understand CPU instructions, we'd be in serious trouble.
Even if it's cheap, 3 implementations are more expensive than one and then you add the additional task(s) of evaluting them and selecting one to move forward with.
I think we definitely need to have alignment, and documentation to support it. I think this can be at the PRD level, mostly.
For many systems, I'd argue technical documentation to understand how the internals are working can simply be handed off to the robots. Or generated on the fly. And if a requirement on the product level is not met, that can be changed under the hood.
I think it's more important. AI gets a lot right, but sometimes it gets things wrong. The document might be the only human authored piece of text, and it will help future agents see that something is incorrect in the implementation.
Thanks for reading and for the thoughtful feedback!
> Only real criticism I have here, is I would not include any code in a design doc, unless it is really really vitally important.;
Yeah, that's fair. If I were setting guidelines for a large org, I'd maybe discourage code snippets in design docs, as it's hard to know when is too much. At my last company, the dev team was just 3-4 people, and I found it helpful to have little snippets in design docs especially when we're talking about semantics of a new library or how to migrate existing code to a new system.
> Orthogonal to the article, but this line of thinking (including the link to the classic 2000 "Things You Should Never Do, Part I" article[0]) may be worth reviewing in the post-LLM world; for all their flaws, LLMs are spectacular at language-to-language translation, and we already have one major project released[1] that shows porting a relatively large and mature project from one language to another is possible.
Yeah, I agree this could change with LLMs, but I think Joel is still correct up to today. Bun is an interesting case because it's friendliest possible conditions for an LLM rewrite (self-contained inputs and outputs, easy to test old implementation and new implementation side by side, huge test corpus w/ third-party tests). I haven't followed it closely, but it seems like the jury's still kind of out as to whether the rewrite was a good idea.
Author here. Happy to take any feedback about this post.
I learned to write design docs at Microsoft and Google, and I thought they both had good culture around docs that hasn't percolated out as well as other engineering practices at those orgs. I haven't seen a thorough explanation of how to write design docs, so this is my attempt to externalize what I've learned about writing them.
The documentation became the bible, and although I tried to keep the design goals at the conceptual/logical level the following would happen the moment the implementation started:
1. This is ambigous the docs need updating, please refactor your design
2. This doesn't work as the doc stated why did you get this wrong
3. The requirements have changed you need to update it
The burden to get "everything right" was absolutely lumbered of me, and the document became an easy finger pointing exercise, even if blame wasn't intended by those launching the critique the burden of "owning" the doc and the consequences of the doc was real.
Now there's a good chance that I am just a shit documentation writer, I can accept that, but honestly I feel like for the vast majority of organisations this just falls into another step of the waterfall pattern, which just doesn't work.
What you're describing sounds like toxic team dynamics rather than something specific to design docs. Do you work effectively with your teammates outside of design docs, or is there similar tension/hostility everywhere?
What you're describing sounds like the design process working as intended (modulo the finger-pointing). The design doc should be unambiguous, and the implementation should match it.
Assuming this isn't just symptoms of a sick team, my other explanation is that your teammates find your deviations from the design doc unexpected. It sounds like you're running into situations where you can't implement the design doc as written, so you're proactively making your own design choices and showing your teammates the implementation. Could you loop your teammates in earlier on before you've implemented the code? Like, "The design docs says we're supposed to use SQLite, but I realized that SQLite doesn't support types the way we expected, so I think we should switch to Postgres for X, Y, and Z reasons."
Thanks for taking the time to reply, please take my response as earnest attempts to better myself :)
So, I think I didn't elaborate on the process enough to get the answers I was looking for. Here's what happened.
1. I would uncover a requirement from the business
2. The conceptual design would be written, high level understanding of the feature etc that was trying to be written
3. Conceptual design would be signed off
4. Logical design would take place ( I think this is basically everything from the Constraints section down in your model )
5. Logical design would get reviewed and signed off.
Now technically everyone could and was encouraged to sign off the logical design, but in reality maybe one other person in the team would, I don't really know the reasons why.
Then implementation start, this was usually NOT me but another team member tasked with the design, and this is where the process really started to fall down with the onus being put squarely back on me as to why the design didn't work :)
I also actually tried the other approach, implement as much as possible ( because AI fast ) and then reverse engineer the design, but then that very much felt like, what the hell is the point now? I might as well just use the standard code review process.
It's hard to say without knowing what things are like at the company/team you work for, but I can say the things you're describing sound unusual to me.
Most significantly, it's strange for the person writing the design doc not to be the person implementing the code. This is asking for trouble because there's a principal-agent problem[0], and also there's bound to be signal lost in the handoff between designer and implementer. It's not so unusual for the design doc author to work with a team on implementation, but they'd still be actively involved in implementation, which sounds different from what you're describing.
I've also never heard of this separation between a conceptual design doc and a logical design doc. I've been on teams where the product manager writes a UX-focused spec, and then the dev writes a technical-focused spec, but I've never heard of a conceptual vs. logical spec.
Does the org have a strong engineering culture in other ways? Like automated tests, automated deploys, automated monitoring/alerting, useful code reviews? Because the easiest answer is that you're in an org with poor software engineering practices, or at least weak documentation culture, and the design review process you're experiencing is there for historical or political reasons rather than engineering reasons.
Well, there's a fascinating insight, most orgs I've ever worked for have had the documentation produced by someone else and them implemented by the developer.
For instance the enterprise architect/solutions architect ( or whatever job title is fashionable at the time ) would write the document and then send it over to the development team and obviously that wouldn't work.
I actually adopted the conceptual/logical design from two ex Hewlett Packard engineers/architects who introduced the idea to me, probably some twenty years ago now!
Yep we're strong in many other ways, but finding the transition to AI challenging :)
Reviews have somewhat turned into, I send you the output from my AI agent of choice, and you tell your AI agent of choice to fix it :)
Thanks again for the time taken to reply :) It is appreciated.
This is good guidance, but what do you have to say about convincing your team of developers to live it out?
I've found that developers usually like writing code and avoid contributing to documentation. For some, it's actually scary because (edit: for them,) high quality writing is harder than high quality coding, and it can be avoided quite a bit.
On the project side, it's rare for the implementation and verification stages to not consume all the budget and more, and delivery creeping past the original optimistic date. So there's no time or money to spend on documentation.
The combination is that even with your great advice in hand, it's hard to navigate to really solid and comprehensive design documentation underpinning the products.
This is a good question, and I have a super long answer that's been in my head for like 8 years about how to influence your teammates to adopt good engineering practices.
The short answer is that most useful software engineering practices are a risk to the first person on the team to adopt them. For example, if everyone on your team thinks automated testing is stupid and you adopt automated testing, it will look like your work is worse because you're slower in the short-term, and maybe you have to do even more work when teammates break your tests.
It comes down to accruing social currency with your team. Your teammates don't want to take a risk for you if you have a history of bad ideas that wasted everyone's time. But if, for example, you implemented automated deploys to replace a tedious workflow developers had to do manually, people would see how your ideas have payoff, and they're more willing to invest a little bit if they expect ROI long-term.
When I've convinced my teammates to invest in design docs, I made sure I had some wins under my belt before I started pushing for everyone to write design docs. I invested a lot in docs myself so my teammates could see the value before I asked them to start writing.
This is also a place where you have to think about politics a bit. Documentation has a much better shot if it has support from the top, so think about the pitch to your manager or dev lead about how design docs make their jobs easier.
Good read, although the document would be very lengthy if I am to write all the sections in the articles. I sometimes break down the design doc into multiple design docs.
- Manager doc : Summary(Background + Objective), User Story (Scenarios), High level architecture, open questions, task break down + timeline including other teams' tasks
- Engineering Architecture doc: Summary, Glossary, Goal (Functional + Non-functional + Non-goals), More detailed architecture & components between, open questions, tasks break down + timeline
- Engineering API / Database design doc: Similar summary + link to architecture doc. More detail information on API (eg: input params, output params, example client code) + database design (eg: database type + fields), open questions
Each doc is to be read within a single meeting. The shorter doc helps narrowing down the discussion scope.
Disclosure: I worked at Amazon where there is a typically 1 hour meeting session with the first 15-30 minutes dedicated to reading. It's probably why I multiple short docs over a single design doc. Telling people "Today, we'll read section 1,2,3,5,8 of the doc" didn't really work.
reply