> If your configs are really complicated enough that they need to be generated at the time your main program runs, to me that means your code needs refactoring.
I'm not sure that's true. In fact, I would go so far as to say for some programs, it's the opposite.
In the example I describe above (forgetting about lambda functions or anything like that) I'm shipping a program that consumes a small set of very rigid input options. I consume a single array of strings that need to resolve to file paths.
That's incredibly easy to validate: that's the type of limited input where I can be pretty certain that my program will not have any bugs or vulnerabilities around the input. It's also an incredibly simple API to teach to users, it's literally one option. And importantly, my program is still pretty powerful -- you can do whatever you need to do to get that file list without me needing to extend what input I accept.
I can't think of a good way to convert that to a pure JSON/YAML config without either making my software more error prone or making my software much less useful.
If returning lambda functions from a config file scares you, forcing the config to be a static object literal but allowing that object literal to be generated at runtime will still (often) result in a runtime that is less error prone, easier to validate, and easier to debug when something goes wrong. This is because at the end of the user's dynamic config, they'll still be returning a small, easy-to-understand object literal -- whatever logic was used to build it.
If instead you put that logic in your main program, and you accept globs for files, and you start doing complicated things like allowing arrays of multiple globs and file excludes -- well, suddenly you have a big chunk of parsing logic in your main program that end-users can't debug when something goes wrong. It becomes harder to validate whether or not you have some bug around the order that patterns are passed to you, or whether there's an error if you try to use a globbed filepath as an excluded filepath, or whatever.
In other words, it is usually better to use procedural logic to generate declarative logic than it is to use declarative logic to generate procedural logic. Of course, your program might be simple enough that you can get away with a config file that's turned into a bunch of `if` statements when its parsed. But it probably would be simpler if you got rid of as much parsing code as possible and focused on more rigidly validating a smaller, stricter set of options.
> I can see providing a separate config editor program users can use to generate configs for your main program
If you ignore the stuff about lambda functions, that's exactly what we're doing. We're just piggybacking on a real language to do it instead of wasting time writing our own graphical/command-line utility.
----
Where returning functions is concerned:
> Either that, or what you are calling a "config" is really a plugin/extension/scripting system
I think this is a kind of blurry line. I'm not going to say there is no difference between a plugin system and a config, but I have run into programs that have both configs and plugin systems, and where the plugin system exists mostly to deal with the fact that the config API is not very useful.
As a user, given the choice between learning one API and two, I usually prefer to learn one API. A lot of static site generators come to mind; I don't want to learn two different APIs for a templating engine just so I can change how links render. I don't want to learn both your plugin API and your config API. Just give me one, single API for templates that can be fully described in a single paragraph of text.
I dunno. I've been doing functional programming long enough that the idea of lambdas just doesn't scare me. Functional paradigms are wildly useful when building library APIs that get imported; I don't see why we should throw them away just because our API lives in a dotfile.
> I consume a single array of strings that need to resolve to file paths.
And I'm finding it very hard to imagine why your users would need to write code to give your program those paths, instead of just giving you a text file with a list of file paths in it. Of course I don't know anything about your program or its use cases other than what you've said here. But I do know that if your users are writing code, what you're describing doesn't look to me like a config. See further comments below.
> that's exactly what we're doing
Making your users write actual code is not the same as providing them with a config editor.
> I think this is a kind of blurry line.
I think the line is pretty clear: are your users expected to know how to write code, or not?
If they're not, then why would they need the ability to write code to generate your config, since they can't do it anyway?
If they are, then as I said, you don't have a static program with a config, you have a scripting language and an interpreter. Your program is just a customized version of the standard interpreter for whatever programming language your users are expected to use.
I don't have any problem with this if it's appropriate for your users. I just don't think the word "config" is the right one to use to describe it.
> I've been doing functional programming long enough that the idea of lambdas just doesn't scare me.
Whether it scares you or not is irrelevant unless you are the only one writing configs for your program. If your users are doing it, the relevant question is what scares them, not you.
Sure -- if your users can't write code, then you obviously can't tell them to use code to generate a config.
But take a step back and look at a lot of the examples that people are bringing up here -- Docker, Babel, Webpack, Github Actions, code linters, testing libraries, Ansible. These are all environments where it's pretty reasonable to expect users to know how to write code.
If you're mostly getting hung up about me calling this a config, let me see if I can rephrase the original article's point in a way that is more palatable to you:
A lot of people are building programs explicitly for programmers that use config files, when those programs would be much better suited by instead dropping the configs entirely and using something more like scripted options.
These program authors struggle building a config schema that is powerful enough for programmers yet simple enough that programmers won't get irritated learning yet another multi-page API. What the authors have missed is that they shouldn't be trying to control their program with a config at all. Their entire product is geared at programmers, so they should just let programmers generate the options that they pass in.
There's very little reason for Babel to be setting options with a JSON config file instead of a JS script, because nobody uses Babel other than Javascript programmers. For these specific programs that are struggling to express themselves with YAML/JSON, a non-config scripted setup would be less error-prone and easier for their users to learn.
Keep in mind that config files are not just about writing them. They are also about reading, and in my personal experience, when I encounter a heavily scripted config file in some javascript project, my inclination is to flatten it out to a plain JSON file just so I can see what it's actually doing.
This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles. The problem with programmable scripted config isn't the writing of it. it's the reading and maintenance. Programmed config is a layer of obfuscation, that I, as a maintainer, need to spend time and mental effort decoding to get work done.
I'm sympathetic to this, and I suspect the original author is as well (see their comments on Emacs). I think you're making a very good point that people need to keep in mind when they write configs.
However, very specifically where Makefiles are concerned:
> This is the beginning of the logic that lead to things like Automake, and completely impenetrable makefiles.
The problem with Make in specific isn't scripting the build, it's that Make is a completely separate domain-specific language that isn't particularly well structured or particularly elegant in the first place.
The comparison I would draw is with Grunt.
Grunt is also a programmable config that allows you to do powerful stuff. It's also IMO a giant disaster and most projects shouldn't use it. Grunt is using programming to control how a project builds, but it's also tying tons of layers of abstraction into this secondary API that's unreasonably difficult to debug, and that lends itself to really bad, unnecessarily verbose control flows.
I was teaching interns how our build process worked at a company, and I had an "aha" moment when I realized that in the time I could teach them Grunt's API, I could also teach them how to write a maybe 100 line pure-JS script that did literally everything Grunt was doing for us.
When I look at Makefiles and Gruntfiles and cringe, very often the thing that makes me cringe are these giant pillars of abstractions and control flows of "program X calls program Y and pipes its output to program Z, and good luck intercepting that specific output to see what's going wrong".
There's something to be said for getting rid of these giant build systems that are spanning multiple processes and languages in general, and instead doing something very direct in one language in one file. And I kind of think that's a concern that cuts across different config types. At my current job, we're in the process of dropping Team City, because we realized that for our needs, we can get by with a single 30-ish line bash script that gets copied to each build machine and auto-run every 5 minutes. And Team City is using a lot of declarative options in a very structured web UI, but the Bash script is still a lot easier to debug and reason about than Team City's whole... thing.
Which is not to say you're wrong, I do think you're making a completely valid point. But I don't think the obfuscation you're talking about is inevitable, I think that's just a side effect of programmers thinking they're clever and trying to make generalized build systems for specific tasks.
it comes down to what exactly is the configuration for. Is it a few settings (server urls, ports, table names, s3 buckets, directory paths, usernames and passwords), or are you actually scripting the tool? Build systems almost straddle the two worlds - they're kind of somewhere between config and scripting. At some point you gotta make the call: if the configuration file is hard to write, check first if it's a problem with the tool the configuration is for. then second, check if what you're actually doing is programming, not configuring. This is repeating what others have wrote, but I solidly agree- at some threshold of complexity you've got to stop calling it "configuration", and start calling it a greenspun's 10th:
Specific to your comments on Grunt: mostly agreed w your analysis; IIRC, I felt it was part of why Gulp kind of ate Grunt's lunch when it came on the scene. But that was a long time ago.
This is exactly why Gulp was such a refreshing change of pace from Grunt. Gulp was "just JS" APIs, and made it _so much easier_ to understand what the heck was going on during build.
> These are all environments where it's pretty reasonable to expect users to know how to write code.
For those kinds of environments, yes, I agree that it makes a lot more sense to just have users write code in the same language that the main program is in, than to try to hand-roll a config file format and structure.
I didn't get the sense from the article that its scope was limited to these kinds of environments, but I may have missed something.
move over templated yaml files, get ready for templated python files.
I heard you like config files, so I set up a config file for your config file so you can config while you config.
You might think you are joking, but that's exactly what happened with sendmail. It's configuration file is so complex (and I think Turing complete) that people came up with a completely different configuration file to generate the configuration file.
Real talk though, at the point where people are building templating engines for YAML files, doesn't it become kind of obvious that YAML isn't powerful enough or expressive enough for at least some of the projects that are using it?
We've got a project in Python, but Python is too hard to read, so instead we have YAML. But YAML is too hard to write, so now we have YTT.
We've now taken a program that was written in one language, and magically transformed it into 3 languages instead :)
> We've got a project in Python, but Python is too hard to read
You said the kind of scripting system you were describing was for cases where your users can write code.
If your users can write code, I don't see why Python would be too hard to read.
If your users can't write code, then, as you agreed, you shouldn't expect them to, which means the kind of system you were describing won't work. And that means you should rewrite your code so it doesn't need such a complicated configuration that it seems like writing code is necessary to express it.
Yeah, I was trying to joke -- if your users are OK with Python, then just let them use Python.
What I was getting at was that this is a scenario where concerns over Python users needing to actually use Python has morphed into those same users now needing to use 3 languages.
If you're templating YAML, it just means that you don't know that you can generate YAML.
Dev ops is a very different world. Yes they can script, but that doesn't mean they will modify any part of your program, even a python config file directly. They will template it.
If you find yourself thinking that your config language isn't powerful enough, the solution is to either generate it, or to fix the program to not require a complex config file. There is a long term, 10 year cost to making config turing complete that I don't believe is worth paying. It'll save you time up front, maybe, but I certainly am not going to be impressed with having to modify your program just to "configure" it.
Finally, given that's the environment we're dealing with, from now on the rational thing to do would be to design config file formats expecting them to eventually get templated. Because that's how they get automated.
> Finally, given that's the environment we're dealing with, from now on the rational thing to do would be to design config file formats expecting them to eventually get templated.
But... that's kind of what a dynamic config file is.
If you're writing a JS program that imports and runs a JS file to generate an object literal for its options, you have just made a config file format that you expect to be generated, or templated, or whatever you'd like to call it. The only difference is that you're using using 1 language instead of 3, and your users don't need to have a separate step during build to generate the file.
People keep on circling around to, "but what if my user doesn't know the language or isn't comfortable with it?" And sure, if your user isn't comfortable programming, don't do this. But if your user does know how to program, don't force them to learn a new templating format, just let them write the code that they already know.
Or honestly, just play the joke out straight and build a template engine for your source language. I get that you're making a joke above, but a templating engine that spits out valid Python/JS code from valid JSON/YAML is trivial to write.
var template_source = fs.readFileSync(process.argv[1]);
var output = JSON.stringify(JSON.parse(template_source), null, 2); /* check malformatted input and prettify */
fs.writeFileSync('config.js', `module.exports=${output};`);
So if you go this route and a few of your users end up wanting to template JSON/YAML anyway, then fine, they still can.
great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file.
or put another way, stop trying to reinvent the wheel and expect that OTHER people will script the generation of the config file, in their own way using their own tools and languages, and just make that easier for them.
when devops are involved, the question isn't whether they're comfortable with whatever language, the question is how easy it will be for them to template the file, because they simply will not learn your special configuration language, whether they possibly could or not.
The point isn't about whether we should make the automation people learn a new templating language. the point is they simply will not. they will find where the relevant parts of the config file are, and they'll stick a ${my_variable} in there, process it using their own tool, whatever it is, regardless of how cleverly (or not) you designed the scripting language around the config. And now whether your config file is python yaml or json is just irrelevant. They will not use python to loop through a file list, even if they could, or know how. They will template that. all you've done by switching your config to python is create a breeding ground for script injection attacks.
So ultimately it's the wrong question. What I'm proposing is, design our config file expecting the ${subsititution_variable} will get edited in at some point, and you guard against script injection assuming that is comimng.
> great now integrate that in all the automation tools that devops use, and train them how to use the integration instead of just templating the config file.
I don't understand what you're getting at. This is just a 3 line executable you call when you're done with whatever templating process you want to use. Ship it with the program.
Now your engineers can stick a `./config.json` in and template it with whatever automation tool they want in whatever integrated pipeline they want to work with. They never need to acknowledge that runtime option generation is possible. They never need to run even one extra shell command.
So I just don't see this as a real problem. If your config script is ultimately returning a static object literal then there is no special knowledge with this setup that anyone needs to learn. All we're doing is giving people the option to script a config on a lower level in a real programming language. We're not inventing a new scripting language or toolchain. And it's still trivial for engineers to ignore that if they want to compile a config with a 3rd-party templating tool.
> What I'm proposing is, design our config file expecting the ${subsititution_variable} will get edited in at some point, and you guard against script injection assuming that is coming.
I would maintain that a config file isn't really where you should be handling security, that's something that should be ingrained into your program at a deeper, more fundamental layer.
But granted -- if for some reason your application can't be run in a secure unprivileged environment, and you need to accept untrusted config files within that environment, and you also can't run the config generation code in a secure unprivileged environment, then it would make sense to strictly limit a config file to a static format like JSON, where you know that no one else can inject actual code. But that's a very different concern than usability, and affects an even smaller subset of software than what we were talking about above. This is just not relevant for systems like Github Actions, Babel, Webpack, or Ansible, all of which are built on either the assumption that the config is trustworthy, or that the trustworthiness of the config doesn't matter because of sandboxing.
In most (not all, but most) cases, if you don't trust your user config, you're already at the point where you should be running your software in an unprivileged sandbox.
Yes of course you're right about all that. My only point is, it's a fact that devops add templating tags to YAML files. This is not theoretical. The templating tags are likely not escaped properly, so, aside from the worst case scenario of script injection, the more innocent situation of causing parsing problems is possible and likely.
It's worth very seriously asking the question of WHY devops are doing this instead of just generating the yaml file with code, which from our perspective as developers would be easier and more sane, and allow us to use the full power of a programming language. I think it's a more important problem to solve than ergonomics of config for the original developers, because end of the day, who will need to interact with this configuration file more? Developers or the people responsible for deploying the application?
Here’s a real-world example: clangd, the language server, takes a file with the compiler flags it should use for completion. clangd works with C, C++, Objective-C, and Objective-C++; obviously, I would like to use different flags for each. Currently I get around this problem by running four “instances” of clangd, one for each language, each with its own configuration file. I wouldn’t need to do this if the configuration file was a script that could dynamically generate flags by looking at the source code and inferring what flags would be appropriate.
I'm not sure that's true. In fact, I would go so far as to say for some programs, it's the opposite.
In the example I describe above (forgetting about lambda functions or anything like that) I'm shipping a program that consumes a small set of very rigid input options. I consume a single array of strings that need to resolve to file paths.
That's incredibly easy to validate: that's the type of limited input where I can be pretty certain that my program will not have any bugs or vulnerabilities around the input. It's also an incredibly simple API to teach to users, it's literally one option. And importantly, my program is still pretty powerful -- you can do whatever you need to do to get that file list without me needing to extend what input I accept.
I can't think of a good way to convert that to a pure JSON/YAML config without either making my software more error prone or making my software much less useful.
If returning lambda functions from a config file scares you, forcing the config to be a static object literal but allowing that object literal to be generated at runtime will still (often) result in a runtime that is less error prone, easier to validate, and easier to debug when something goes wrong. This is because at the end of the user's dynamic config, they'll still be returning a small, easy-to-understand object literal -- whatever logic was used to build it.
If instead you put that logic in your main program, and you accept globs for files, and you start doing complicated things like allowing arrays of multiple globs and file excludes -- well, suddenly you have a big chunk of parsing logic in your main program that end-users can't debug when something goes wrong. It becomes harder to validate whether or not you have some bug around the order that patterns are passed to you, or whether there's an error if you try to use a globbed filepath as an excluded filepath, or whatever.
In other words, it is usually better to use procedural logic to generate declarative logic than it is to use declarative logic to generate procedural logic. Of course, your program might be simple enough that you can get away with a config file that's turned into a bunch of `if` statements when its parsed. But it probably would be simpler if you got rid of as much parsing code as possible and focused on more rigidly validating a smaller, stricter set of options.
> I can see providing a separate config editor program users can use to generate configs for your main program
If you ignore the stuff about lambda functions, that's exactly what we're doing. We're just piggybacking on a real language to do it instead of wasting time writing our own graphical/command-line utility.
----
Where returning functions is concerned:
> Either that, or what you are calling a "config" is really a plugin/extension/scripting system
I think this is a kind of blurry line. I'm not going to say there is no difference between a plugin system and a config, but I have run into programs that have both configs and plugin systems, and where the plugin system exists mostly to deal with the fact that the config API is not very useful.
As a user, given the choice between learning one API and two, I usually prefer to learn one API. A lot of static site generators come to mind; I don't want to learn two different APIs for a templating engine just so I can change how links render. I don't want to learn both your plugin API and your config API. Just give me one, single API for templates that can be fully described in a single paragraph of text.
I dunno. I've been doing functional programming long enough that the idea of lambdas just doesn't scare me. Functional paradigms are wildly useful when building library APIs that get imported; I don't see why we should throw them away just because our API lives in a dotfile.