feat: Adds tier attribute to stream processor resource and datasource #3969

kpatel71716 · 2025-12-09T11:30:27Z

Description

CLOUDP-359071

This PR does 2 things:

Switch the admin api for starting an Atlas Stream Processor to use the stream processor startWith endpoint instead of the start endpoint. This does not reduce any functionality for the user but allows us to do the next point.
Add a tier attribute for the stremprocessor resource and datasource. This attribute can be used to start a stream processor on a given tier rather than forcing the processor to default to the workspace setting

Type of change:

Bug fix (non-breaking change which fixes an issue). Please, add the "bug" label to the PR.
New feature (non-breaking change which adds functionality). Please, add the "enhancement" label to the PR. A migration guide must be created or updated if the new feature will go in a major version.
Breaking change (fix or feature that would cause existing functionality to not work as expected). Please, add the "breaking change" label to the PR. A migration guide must be created or updated.
This change requires a documentation update
Documentation fix/enhancement

Required Checklist:

I have signed the MongoDB CLA
I have read the contributing guides
I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
I have added tests that prove my fix is effective or that my feature works per HashiCorp requirements
I have added any necessary documentation (if appropriate)
I have run make fmt and formatted my code
If changes include deprecations or removals I have added appropriate changelog entries.
If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Further comments

github-actions · 2025-12-09T12:05:00Z

APIx bot: a message has been sent to Docs Slack channel

QuiqueSanMongo

LGTM!

lantoli · 2025-12-10T06:56:13Z

internal/service/streamprocessor/resource.go

+			startWithOptions.SetTier(plan.Tier.ValueString())
+		}
+
+		_, err := connV2.StreamsApi.StartStreamProcessorWith(ctx, projectID, workspaceOrInstanceName, processorName, startWithOptions).Execute()


Create function is getting very long, consider extracting some parts into functions

I'll create a follow-up ticket to explore this. I believe we have an existing action item on our end to go through this api flow but I'll reply back here

.changelog/3969.txt

lantoli · 2025-12-10T06:59:50Z

internal/service/streamprocessor/resource_schema.go

 			},
+			"tier": schema.StringAttribute{
+				Optional:            true,
+				Computed:            true,


is it Computed, because Atlas will return the value if not provided by client?

Yes that is correct. Today each stream processor has a tier set even without user input. And we have made changes to return the tier. But now the client can provide the tier to override that

lantoli · 2025-12-10T07:00:23Z

internal/service/streamprocessor/resource_schema.go

+				Computed:            true,
+				MarkdownDescription: "Selected tier for the Stream Processor. Configures Memory / VCPU allowances. Valid options are SP2, SP5, SP10, SP30, and SP50.",
+				PlanModifiers: []planmodifier.String{
+					stringplanmodifier.UseStateForUnknown(),


q: why are we using UseStateForUnknown?

Since this is optional to provide, my thought was that if a user removes the tier from the config, we should not show a plan change. If I have unnecessarily used or missed UseStateForUnknown, I can remove it

For our apis, when a user stops/starts a stream processor, we always use the last tier that the stream processor was running on unless the user explicitly provides an option to override it i.e. we won't go back to the workspace default unless the user explicitly specifies to do so

internal/service/streamprocessor/resource_test.go

lantoli · 2025-12-10T07:07:28Z

internal/service/streamprocessor/resource.go

-			},
-		).Execute()
+		startWithOptions := &admin.StreamsStartStreamProcessorWith{}
+		if plan.Tier.ValueString() != "" {


It looks like you're only changing Create, can the attribute be Updated?

If it can't, it should have the customplanmodifier.CreateOnly() plan modifier. With the current implementation, I think that if customer tries to change the tier value, they will get an "inconsistency error" because the updated value won't match the initial value

can the attribute be Updated?

If it can, would also consider how it behaves if a stream processor is configured as started vs stopped when modifying tier.

The attribute can be updated. I had a couple of checks that I updated and added tests for the same. I believe the logic is complete for Create and Update now

lantoli · 2025-12-10T07:09:25Z

docs/data-sources/stream_processor.md


 **NOTE** When a Stream Processor is updated without specifying the state, it is stopped and then restored to previous state upon update completion.
 - `stats` (String) The stats associated with the stream processor. Refer to the [MongoDB Atlas Docs](https://www.mongodb.com/docs/atlas/atlas-stream-processing/manage-stream-processor/#view-statistics-of-a-stream-processor) for more information.
+- `tier` (String) Selected tier for the Stream Processor. Configures Memory / VCPU allowances. Valid options are SP2, SP5, SP10, SP30, and SP50.


can tier change at some moment, e.g. Atlas decides to scale it up/down? Or it'll always have the value that client used to create the processor? (or alternatively the value decided by Atlas if not sent by client in Create)

As of today, the tier cannot change. It is always the value that the client has set.

It is worth noting that in the future, this could not be the case when we add autoscaling. When that happens, the idea is that the flag to allow autoscaling will not be allowed to be used with this defined tier field. But I am still discussing that with the team

docs/resources/stream_processor.md

github-actions · 2025-12-18T00:28:43Z

This PR has gone 7 days without any activity and meets the project’s definition of "stale". This will be auto-closed if there is no new activity over the next 7 days. If the issue is still relevant and active, you can simply comment with a "bump" to keep it open, or add the label "not_stale". Thanks for keeping our repository healthy!

feat: adds tier attribute to stream processor resource and datasource

e29e4f4

kpatel71716 changed the title ~~feat: adds tier attribute to stream processor resource and datasource~~ feat: Adds tier attribute to stream processor resource and datasource Dec 9, 2025

github-actions bot added the enhancement label Dec 9, 2025

doc: update documentation

64f909b

kpatel71716 force-pushed the CLOUDP-359071 branch from 750d83e to 64f909b Compare December 9, 2025 11:58

kpatel71716 marked this pull request as ready for review December 9, 2025 12:04

kpatel71716 requested review from a team as code owners December 9, 2025 12:04

QuiqueSanMongo approved these changes Dec 9, 2025

View reviewed changes

lantoli reviewed Dec 10, 2025

View reviewed changes

.changelog/3969.txt Show resolved Hide resolved

lantoli reviewed Dec 10, 2025

View reviewed changes

internal/service/streamprocessor/resource_test.go Outdated Show resolved Hide resolved

lantoli reviewed Dec 10, 2025

View reviewed changes

AgustinBettati reviewed Dec 10, 2025

View reviewed changes

docs/resources/stream_processor.md Outdated Show resolved Hide resolved

update docs and address PR feedback

cea1c8f

kpatel71716 requested review from AgustinBettati and lantoli December 12, 2025 12:39

github-actions bot added the stale label Dec 18, 2025

feat: Adds tier attribute to stream processor resource and datasource #3969

Are you sure you want to change the base?

feat: Adds tier attribute to stream processor resource and datasource #3969

Uh oh!

Conversation

kpatel71716 commented Dec 9, 2025

Description

Type of change:

Required Checklist:

Further comments

Uh oh!

github-actions bot commented Dec 9, 2025

Uh oh!

QuiqueSanMongo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Dec 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants