Allow JSON files as input for Secrets Stores and KVStores

[GeeWee]

Currently the fastly.toml specification allows you to pass in a JSON file for config stores (and earlier, also for dictionaries)

Something like that is extremely handy to be able to do, to e.g. share configuration across multiple fastly.tomls for environments, autogenerating these values from other scripts, not having to check them into CI etc.

Is that something you might be interested in adding back?

[fgsch]

This would be a great addition. Would you be interested in opening a PR for this?

[GeeWee]

I think I could try to find time for that if it would end up being accepted?

[fgsch]

@GeeWee I'm pretty sure that'd be the case 👍

[GeeWee]

Okay, can you check? I'd hate to do the work and then have it languish :)

Do you have a proposal for the user-facing API in the TOML file? I'd like to lean against e.g. ConfigStores, but I think that's slightly harder, as that uses the format: "inline-toml" for keys defined inside the toml file.

[fgsch]

Sorry, looking at Viceroy I can see support for the file stanza for the secret store (#210) and kv store (originally added as path in #167 and later renamed for consistency in #206).

Is this what you wanted?

[GeeWee]

I don't think so based at the documentation and the PR. I want to be able to specify a file with all my key-values, so e.g. a JSON object or similar, while the file property currently just allows me to specify the contents for a specific key - if I understand correctly?

[fgsch]

Ok, if you want all your key-values, that might require a bit of internal discussion as it is currently not part of the spec.
Sorry for the initial misunderstanding.

[GeeWee]

Any way I can push on that discussion, or if you have an idea about a timeline? This would bee extremely handy for us, and I was maybe a little too eager, so I already did the work to implement it in #365 😅

[kpfleming]

As I noted in the PR, we're just about to embark on a process to redesign the schema for fastly.toml to make this sort of thing easier to do (and make it more flexible as well), so I'd like to hold off a few weeks before we continue this discussion to see how this feature fits with that. Once we have an initial proposal we'll post it on our community forum (https://community.fastly.com) to get input from you and others who use Viceroy, and I'll update this ticket when that happens.

[joeshaw]

Related to (but distinct from) #359

[GeeWee]

@kpfleming do you have a rough timeline for this redesign?

[kpfleming]

There should be a post on our community forum in the next day or two, apologies for it taking much longer than planned to get the process started.

[GeeWee]

I'm unable to find the community forum post - any chance you have a link handy? :)

[kailan]

Hi @GeeWee –– apologies for the wait on this. I've just opened a community forum thread where we would welcome your input on fastly.toml: https://community.fastly.com/t/have-your-say-on-the-future-of-fastly-toml/2785

[GeeWee]

Thanks! Just gotten back from vacation - I've written my thoughts there. I assume there's no timeline for when this will change, and that the PR I've proposed won't be accepted until then?

[kpfleming]

We're aiming to close out the feedback/input process this week, and then draft a proposal for what the new format might look like... which will then lead to a comment/review period. Since that will take some time and your PR is relatively small, I'll work on getting it merged soon.

[kpfleming]

This is now partially complete, as the PR for JSON support for KV Stores has been merged, and the feature will be included in the next release.