feat: activityRelations buckets for subset of projects

[epipav]

Note

Introduces 10-way bucketing for activityRelations, adds routing/union and enrichment/snapshot pipelines, and refactors most analytics pipes to use the new bucketed datasets, plus new monitoring and minor datasource tweaks.

• Data infrastructure (bucketization & snapshots):
  • Add 10 materialized bucket datasources for activityRelations (activityRelations_bucket_MV_ds_0..9) and 10 cleaned/enriched bucket datasources (activityRelations_deduplicated_cleaned_bucket_0..9_ds).
  • New pipes for bucket MVs, on-demand snapshots, and per-bucket clean/enrich copy jobs; introduce activityRelations_bucket_routing and activityRelations_deduplicated_cleaned_bucket_union.
  • Add enriched datasets and snapshot flow: activityRelations_enrich_snapshot_MV(_ds), activityRelations_enriched_deduplicated_ds, and a daily snapshot merger.
• Analytics refactor (use buckets):
  • Switch core filters and many widgets/pipes (active contributors/orgs/days, counts, cumulative, retention, geo, leaderboards, health score, org dashboards, insights copy, issues/PR analysis, packages, search, etc.) to read from routing/union bucket sources.
  • Update PR analysis baseline/initial pipelines to use new enrichment MV and datasets; add new review_efficiency pipe.
  • activities_daily_counts now reads from activityRelations_enriched_deduplicated_ds with latest snapshot.
• Monitoring & ops:
  • Add monitoring pipes for copy job spread, entity row counts, and long-running endpoints; add sitemap pipes.
  • Update format_all.sh to support --sequential mode.
• Datasources & misc:
  • Add activities_backup_consistency_audit datasource; add/adjust partition keys/TTLs on several tables.
  • Minor fixes (e.g., reviewer activity type mapping).

^{Written by Cursor Bugbot for commit cafbdde. This will update automatically on new commits. Configure here.}

[cursor]

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on December 12

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

Conventional Commits FTW!

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.