add process and guidelines

[LiorLieberman]

/kind documentation

What this PR does / why we need it:
Add process, guidelines and guardrails

Does this PR introduce a user-facing change?:

NONE

[guicassolato]

Thanks for putting this together, @LiorLieberman. I think we really need this. We obviously want to avoid any heavyweight process, but a minimum set of guidelines to govern how the project is meant to operate, always in a safe and just manner, is a good thing IMO.

A couple of questions I believe this doc could try to answer:

• How to decide when a proposal is ready to start being prototyped
• How to distinguish interventions on a proposal that are meant to improve the quality of the proposal, from a technical point of view, from the (often inadvertently) bad ones
• How to avoid proposals being placed from start to favour a single implementation, with least chance of later becoming a standard
• How to mitigate our own bias as maintainers of the project when reviewing a proposal

[LiorLieberman]

How to decide when a proposal is ready to start being prototyped

I think this is something that the prototype itself can also tell. We should probably lean into the group of contributors we have as well. But not really why do we need something to say "green light -- now you can protoype". Instead we should be relying on code reviews when for the protoype PRs. Remember at this stage we dont release anything.

How to distinguish interventions on a proposal that are meant to improve the quality of the proposal, from a technical point of view, from the (often inadvertently) bad ones

(a) I dont think this have a solution generally. (b) I dont think we are at a point where this needs definition. I hope we are not at a point where we believe someone has bad intentions. The project is very new, I dont think we need additional red-tape especially given that we narrow it down so much to say, no releases.

How to avoid proposals being placed from start to favour a single implementation, with least chance of later becoming a standard

Code review. Nothing is becoming a standard at this stage. We agreed that we are at the very early stage of the project and for something to become a standard it needs more implementations, and will probably be after a few iterations of the protoype. This does not mean that we need to accept whatever, but it does mean that we need to be more agile and iterative here and not trying to get everything perfect from start.

How to mitigate our own bias as maintainers of the project when reviewing a proposal

Same as number 2, I dont have a good solution here, and again, we have not faced this problem (yet). Lets solve things as we face them. Some (proven) ideas - Code review, multi-vendor representation, etc.

[david-martin]

To Gui's point about when to protoype, we could add some additional guidance here.
For example, if there are questions in a proposal about if something is feasible or scalable, or it isn't clear or obvious how a worked example is meant to actually behave, a proposal could be kept open until those questions or concerns are answered via research and/or prototype.
Alternatively, if a proposal is generally clear on the intent, it can be merged and a prototype done at that point.

[bowei]

I agree that the bias should be towards demonstration via the reference implementation. This is the one of the best ways we have towards shaking out what makes sense.

[shaneutt]

At a high level I am very supportive of an iterative process. I have been advocating for these practices for many years in SIG Network. You can see traces of this, including some prior-art for process around this in the Gateway API GEP Process, which includes a Prototyping phase.

The fundamental point of the Gateway API Prototyping phase the was that things effectively aren't present in main. This helps reduce contention from contributors who (rightly) expect to have equity in the project's future.

This is, I think, at the heart of why the community has had contention which we're now trying to resolve by adding process: from what I've seen so far there's a strong desire to only do minimal review, emphasizing more (and smaller) PRs. A "fix-it-forward" sort of mentality. This "fix-it-forward" approach could work. However, there are many in the community who have a stake in the future of this technology.

This project (by way of being a kubernetes-sigs project) has (whether you like it or not) established itself as "The definitive future place for Kubernetes standards around agentic networking to emerge". There will undoubtedly be multiple implementations in the long run, and nearly everyone in the community is working on this right now, so many have a stake in this. Additionally, I can't think of anyone off-hand who doesn't think that the main branch of a repository in https://github.com/kubernetes-sigs is "sacred" at some level.

I don't think stopping and trying to add a bunch of process right now is a good use of time, given the objectives and how fast this technology area is developing. You'll still have the fundamental problem that some here want to build something that's not yet for anyone, but put it in a position that signals that its for everyone. I think the important thing is to find the right balance between the desire to quickly iterate, and proper signalling and the community's need for equity. I recommend this for prototyping:

1. Set the proposal status to indicate prototyping
2. Create a non-main branch
3. PR like crazy into that branch, with as little review as you like. Lean, iterative and fast
4. Repeat #3 over the next few weeks until something forms that starts feeling solid and might be ready for wider use
5. PR against main from that branch, in reasonably sized chunks
6. Now: be prepared for a more complete review process, with multiple stakeholders included, and probably some need for compromises. This will take some time, but it's worth it to keep our community strong and integrated.

[robscott]

Thanks for the suggestions @shaneutt! I'm mostly aligned with what you're suggesting here, but I'm not sure why we need a separate branch here. You seem to be taking a similar rapid iteration approach in WG AI Gateway but have not chosen to use a separate branch for this and instead PRs are going directly to main. Why the distinction here?

[shaneutt]

Thanks for the suggestions @shaneutt! I'm mostly aligned with what you're suggesting here, but I'm not sure why we need a separate branch here. You seem to be taking a similar rapid iteration approach in WG AI Gateway but have not chosen to use a separate branch for this and instead PRs are going directly to main. Why the distinction here?

Sub-projects and Working Groups don't really compare. The AI Gateway WG, as a working group, does not own code or APIs. The product of iteration cycles there can only be more proposals to the relevant SIGs and sub-projects. Ultimately we haven't faced any similar contention thus far, we're ready to change our process and try to accommodate if there are concerns. Given your concern, I have gone ahead and created kubernetes-sigs/wg-ai-gateway#18 to use this same branching strategy for any potential prototyping there (the difference being they likely wouldn't ever merge to main, except maybe for posterity before the repository is archived when the WG concludes).

[MikeZappa87]

At a high level I am very supportive of an iterative process. I have been advocating for these practices for many years in SIG Network. You can see traces of this, including some prior-art for process around this in the Gateway API GEP Process, which includes a Prototyping phase.

The fundamental point of the Gateway API Prototyping phase the was that things effectively aren't present in main. This helps reduce contention from contributors who (rightly) expect to have equity in the project's future.

This is, I think, at the heart of why the community has had contention which we're now trying to resolve by adding process: from what I've seen so far there's a strong desire to only do minimal review, emphasizing more (and smaller) PRs. A "fix-it-forward" sort of mentality. This "fix-it-forward" approach could work. However, there are many in the community who have a stake in the future of this technology.

This project (by way of being a kubernetes-sigs project) has (whether you like it or not) established itself as "The definitive future place for Kubernetes standards around agentic networking to emerge". There will undoubtedly be multiple implementations in the long run, and nearly everyone in the community is working on this right now, so many have a stake in this. Additionally, I can't think of anyone off-hand who doesn't think that the main branch of a repository in https://github.com/kubernetes-sigs is "sacred" at some level.

I don't think stopping and trying to add a bunch of process right now is a good use of time, given the objectives and how fast this technology area is developing. You'll still have the fundamental problem that some here want to build something that's not yet for anyone, but put it in a position that signals that its for everyone. I think the important thing is to find the right balance between the desire to quickly iterate, and proper signalling and the community's need for equity. I recommend this:

1. Close this PR
2. Create a non-main branch
3. PR like crazy into that branch, with as little review as you like. Lean, iterative and fast
4. Repeat #3 over the next few weeks until something forms that starts feeling solid and might be ready for wider use
5. PR against main from that branch, in reasonably sized chunks
6. Now: be prepared for a more complete review process, with multiple stakeholders included, and probably some need for compromises. This will take some time, but it's worth it to keep our community strong and integrated.

I am going to +1 one @shaneutt has suggested (and also what I shared in the slack thread). I see main as the authoritative source of truth (fully agreed upon by the community stake holders). If you need to prototype, that can be done in a shared common branch. I have rewritten entire parts of the K8s code and never needed anything in the main branch for example.

[shaneutt]

Talked with Rob on Slack to further explore the branching strategy. Clarification: I think the proposals need to be in main, I don't think you want to have a bunch of proposals hiding out in branches everywhere. That will be hard for newcomers to figure out. Just flag proposals which you're going to go spend some cycles prototyping as Currently Prototyping or WIP or something, and they can link to the branch where the prototype APIs/implementations are being rapidly developed. Then before they mature out of Prototyping into the next phase, do the larger review. That larger review, which includes multiple stakeholders and will inherently take some time, has to eventually happen.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LiorLieberman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [LiorLieberman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LiorLieberman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [LiorLieberman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[LiorLieberman]

( I messed up and closed it accidentally, opened again now)

[bowei]

"We realize this is an experiment that is going to be different than how many Kubernetes projects have evolved in the past. The agent networking space is moving rapidly and it is worth trying a new methodology to keep pace with the developments."

[bowei]

I agree that the bias should be towards demonstration via the reference implementation. This is the one of the best ways we have towards shaking out what makes sense.

[bowei]

Suggest:

• This project may have internal, incremental commits and the contents of the repo (e.g. apis/) should not be treated as canonical towards an API release while iteration is occurring.
• Official releases: the project will explicitly publish releases (e.g. alpha, beta) that adhere to the standard Kubernetes version guarantees.

[bowei]

Nit: I think there is only one "reference implementation" (in this repo).

[bowei]

Nit: Would be good to give end of Feb or beginning of Feb

[david-martin]

I'd recommend early Feb, to give more time for the full reference implementation end of March

[david-martin]

I think the proposals need to be in main

Yes. This is important to avoid confusion on where proposals are, and if there are dependencies between proposals (which is the case with AccessPolicy/Backend and AuthScheme).

Just flag proposals which you're going to go spend some cycles prototyping as Currently Prototyping or WIP or something, and they can link to the branch where the prototype APIs/implementations are being rapidly developed

Agreed, but the branch doesn't need to be on this repo. It probably should be on a fork.
I share a concern with Lior that having to maintain branches and give access on this repo could become a pain.
Let's push that to the prototype implementors to manage.

Then before they mature out of Prototyping into the next phase, do the larger review. That larger review, which includes multiple stakeholders and will inherently take some time, has to eventually happen.

Agreed. PR to main at this point.
Land on main for others to build on.

If an in flight prototype depends on another prototype (which again, could be the case with AccessPolicy/Backend and AuthScheme), let's push it to the implementors to manage that dependency, rebasing etc..
When something lands back on main, it should become easier for dependant prototypes.

How to decide when a proposal is ready to start being prototyped

I don't think a hard and fast rule is needed here.
What ultimately matters is the proposal PR review.
Ideally a proposal is on main when prototyping is ongoing, but a prototype may need to progress somewhat to answer some questions about a proposal in the early stages so people are happy to merge to main. Bigger questions can be deferred to the conclusion of a prototype after that (and PR to main).