-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DISCUSSION] Should flagd support S3-compatible storage providers #864
Comments
Thanks for starting the discussion @agardnerIT. Let's leave this open for a bit to gauge interest. For those who are interested, please add a comment describing your use case and why an S3-compatible sync provider would address it. |
I'm going to close this as there is no immediate interest / need. We can re-open if someone else is interested in future. My usecase was to store / retrieve flag values from cheap object storage like S3 and R2 etc. |
Hey folks, just getting started with OpenFeature and I was very excited to learn of this project, and I suspect there will be many more newcomers in the coming months looking to trial Far from a blocker to getting started, but in-the-box support would certainly help lower the barrier to entry for curious onlookers such as myself. |
Hey @austindrenski, thanks for letting us know you would like an S3-compatible sync. It would be great if we could figure out a clean way to sync with S3 via HTTP polling. That way, we could use the existing HTTP sync and avoid adding additional complexity and dependency. You could probably get it working without any code changes if you made the S3 bucket public, but that's likely not a great long-term solution. I'll look into this a bit. It will likely come down to supporting the authentication mechanism required by AWS. FYI @toddbaert |
It looks like Open Policy Agent has done something similar already. |
I'm going to reopen this discussion. |
That's actually what I tried after posting here earlier and, after some tinkering, I got things working without making the bucket entirely public (i.e. anonymous access within my VPC, no public access outside of it) by using // sharing for future readers, but be sure to consult your friendly
// neighborhood SecOps team before yeeting this anywhere near prod!
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:GetObject",
"Condition": {
"StringEquals": {
"aws:sourceVpce": "vpce-***"
}
},
"Effect": "Allow",
"Principal": "*",
"Resource": "arn:aws:s3:::open-feature/flagd/*"
}
]
}
+1, skimmed through the sync providers earlier and came to the same conclusion of an
Really appreciate you looking into this, on whatever timetable you're able to. I think finding a good answer for how to handle auth for AWS et al. will be important if For example, while we can use S3-based flag definitions with a little bit of work today, it would be a neat option in the long-term to integrate with DynamoDB (or some other general object/document store) for read/write scenarios. (This is presumably far out of scope for now, but the building blocks here are pretty exciting.) |
Glad to hear you had a degree of success! Agreed on all counts. @beeme1mr and I had a brief chat about this, and basically we came to the exact same conclusions, and share the same questions and goals. |
I've created an issue to add s3 support. |
Requirements
Can / should flagd be extended to support s3-compatible storage services.
These would include:
While flagd already supports the Authorisation header, the difficulty seems to be that S3 auth is built from a dynamically generated (HMAC) header. Presumably calculating this would mean enhancements to flagd at the core.
Signature Timeouts
Amazon's documentation states that:
Signatures last for 7 days (meaning flagd would need a restart (to regen the signature) at least every 7 days)
A presigned URL can be valid for a maximum of seven days because the signing key you use in signature calculation is valid for up to seven days.
References
The text was updated successfully, but these errors were encountered: