Skip to content

Commit 9db6c81

Browse files
zong-zhezong-zhe
committed
feat: add blog for flux-kcl-controller
Signed-off-by: zongz <zongzhe1024@163.com>
1 parent 501d8e0 commit 9db6c81

File tree

4 files changed

+341
-0
lines changed

4 files changed

+341
-0
lines changed

‎blog/2024-02-23-flux-kcl-controller/index.md‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
---
2+
slug: 2024-02-23-flux-kcl-controller
3+
title: Implementing GitOps with KCL & FluxCD
4+
authors:
5+
name: KCL Team
6+
title: KCL Team
7+
tags: [KCL, Github, FluxCD, GitOps]
8+
---
9+
10+
## Introduction
11+
12+
In modern software development, GitOps has become an important technology for managing infrastructure and applications. It can significantly improve automation levels, reduce human intervention, reducing the error rate, and improve overall operational efficiency. This approach has been widely used in cloud-native and other fields.
13+
14+
In a previous article, we shared how to use ArgoCD, KCL, and Github to practice GitOps automation.
15+
16+
- [Implementing GitOps using Github, Argo CD, and KCL to Simplify DevOps](https://kcl-lang.io/blog/2023-07-31-kcl-github-argocd-gitops/)
17+
18+
This article will continue to expand KCL's practice in the field of GitOps, integrate with the continuous integration tool FluxCD, and use KCL, GitHub, and FluxCD to provide a specific practice example, detailing how to build and run a GitOps automation process.
19+
20+
### What is GitOps
21+
22+
GitOps is a software deployment model based on Git. It aims to use Git's version control capabilities to manage and automate the deployment of infrastructure and applications. In GitOps, the Git repository is not only the storage location of the code, but also a reflection of the real environment state. Any changes are implemented through commits to the Git repository, and these changes are then automatically synchronized to the production environment.
23+
24+
Through GitOps, you can effectively:
25+
26+
- Enhance collaboration between developers and operations: Developers and operations can work more collaboratively through a unified Git workflow.
27+
- Improve deployment efficiency and security: GitOps simplifies deployment through automated processes, while providing the necessary audit and rollback mechanisms.
28+
- Improve system traceability: Using Git to manage configurations ensures that every change is traceable, enhancing audit tracking.
29+
30+
### KCL & FluxCD
31+
32+
FluxCD is an automated tool that implements the GitOps model, specifically for Kubernetes clusters. It is responsible for monitoring changes in Git repositories and ensuring that the state of the Kubernetes cluster is consistent with the state defined in the repository. The features of FluxCD include:
33+
34+
- Automated synchronization: Automatically synchronize changes in the Git repository to Kubernetes, achieving continuous deployment of configurations.
35+
- Declarative infrastructure: Manage clusters through declarative configuration files, making infrastructure version control more intuitive.
36+
- Security and compliance: Provide more secure change management and audit tracking through Git's branch and PR mechanisms.
37+
38+
KCL significantly simplifies complex Kubernetes configurations through its abstraction and programmability. It minimizes the error rate, allowing developers to detect potential problems in a timely manner during the writing phase, rather than waiting until runtime. This means fewer configuration templates and stronger multi-environment and multi-tenant configuration capabilities, improving the readability and maintainability of configurations.
39+
40+
By using KCL, developers can precisely define the resources required by the application in a declarative manner. Combined with FluxCD, this declarative foundation can promote the implementation of infrastructure as code (IaC), improve deployment efficiency, and simplify application configuration management. FluxCD, as an automated continuous deployment tool, combined with support for KCL, provides a user-friendly visual management interface for configurations.
41+
42+
43+
In GitOps, developers and operations teams can manage application deployment by modifying application and configuration code separately. The GitOps toolchain, such as FluxCD, will automatically synchronize these changes, ensuring continuous deployment while maintaining the consistency of the environment state. If any problems occur during deployment, you can use the GitOps toolchain to quickly roll back.
44+
45+
### Flux KCL Controller
46+
47+
The Flux KCL Controller is a FluxCD Controller developed for KCL, responsible for monitoring the Git repository that stores KCL programs. Through this controller, FluxCD can expand its automated deployment capabilities, continuously monitor and apply configuration files written in KCL.
48+
49+
![flux-cd](/img/blog/2024-02-23-flux-kcl-controller/fluxcontroller.jpg)
50+
51+
With the help of the Source Controller provided by FluxCD, the Flux KCL Controller can periodically check the KCL files in the associated Git repository. Once it detects new commits or updates in the repository, it automatically triggers the synchronization process of the configuration. This means that any changes to the KCL configuration will be detected and automatically reflected in the state of the Kubernetes cluster, maintaining the latest state and consistency of the configuration.
52+
53+
- [More details about Flux KCL Controller](https://github.com/kcl-lang/flux-kcl-controller)
54+
- [More details about Source Controller](https://github.com/fluxcd/source-controller)
55+
56+
## Case Introduction
57+
58+
We still use a Python Flask application and Github Actions as a CI example, using Flux KCL Controller to integrate FluxCD's capabilities for continuous integration.
59+
60+
We split the Python Flask application code and configuration code into two repositories to separate the focus of different roles such as developers and operations teams.
61+
62+
- Source code repository: https://github.com/kcl-lang/flask-demo
63+
- Configuration repository: https://github.com/kcl-lang/flask-demo-kcl-manifests
64+
65+
The overall workflow is as follows:
66+
67+
![workflow](/img/blog/2024-02-23-flux-kcl-controller/workflow.jpg)
68+
69+
- Pull the application code from Github
70+
- Develop and submit the application code to the Github repository
71+
- Trigger Github Actions to compile the application code, generate a container image, and push the container image to the Docker Hub container registry
72+
- Trigger Github Actions to update the KCL-defined Kubernetes manifest files based on the version number of the container image in the docker.io container registry
73+
- Flux KCL Controller monitors the changes in the Git repository and automatically updates the Kubernetes cluster based on the KCL-defined Kubernetes manifest changes
74+
75+
## Steps
76+
77+
### 0. Prerequisites
78+
79+
- Familiar with basic Unix/Linux commands
80+
- Familiar with Git and Github Action usage
81+
- Understand Kubernetes basics
82+
- Understand FluxCD and KCL
83+
84+
### 1. Configure Kubernetes Cluster
85+
86+
- Install [K3d](https://github.com/k3d-io/k3d) and create a cluster
87+
88+
```bash
89+
k3d cluster create mycluster
90+
```
91+
92+
> Note: You can use other methods to create your own Kubernetes cluster, such as kind, minikube, etc.
93+
94+
### 2. Install Flux KCL Controller
95+
96+
- Install Flux KCL Controller in the cluster using the following command
97+
98+
```bash
99+
git clone https://github.com/kcl-lang/flux-kcl-controller.git/ && cd flux-kcl-controller && make deploy
100+
```
101+
102+
For more detailed content about the installation and usage of Flux KCL Controller, please refer to [Flux-KCL-Controller](https://github.com/kcl-lang/flux-kcl-controller/blob/main/README-zh.md).
103+
104+
### 3. Get the Source Code
105+
106+
```shell
107+
git clone https://github.com/kcl-lang/flask-demo.git/
108+
cd flask-demo
109+
```
110+
111+
This is a web application written in Python. We can use the application directory's `Dockerfile` to generate a container image for this application. We can also automatically build the `flask_demo` image through Github CI.
112+
113+
Because we have already completed this part of the work in a previous article, we will not repeat the content here. You can find more about the Github CI in [here](https://kcl-lang.io/blog/2023-07-31-kcl-github-argocd-gitops/#3-get-the-application-code).
114+
115+
116+
### 4. Submit the Application Code
117+
118+
After the application code is submitted to the flask-demo repository, Github will automatically build the container image and push the artifact to the Docker Hub. This will trigger the Github CI process for the flask-demo repository.
119+
120+
![](/img/docs/user_docs/guides/ci-integration/app-ci.png)
121+
122+
### 5. Configure Automatic Updates
123+
124+
After the Github CI process is completed, it will automatically trigger a CI to update the configuration and submit it to the main branch of the flask-demo-kcl-manifests repository. The commit information is as follows
125+
126+
This part of the content has been completed in a previous article, and we will not repeat it here. You can find more [here](https://kcl-lang.io/blog/2023-07-31-kcl-github-argocd-gitops/#5-configuration-automatic-update).
127+
128+
### 6. Use Flux KCL Controller to Monitor the Configuration Repository
129+
130+
We can use the following command to set the Github repo of the Flux KCL Controller to monitor the configuration repository and automatically update the resources in the Kubernetes cluster based on the configuration content.
131+
132+
```yaml
133+
apiVersion: source.toolkit.fluxcd.io/v1
134+
kind: GitRepository
135+
metadata:
136+
name: kcl-deployment
137+
namespace: default
138+
spec:
139+
interval: 10s # Every 10 seconds, check the configuration repository for updates
140+
url: https://github.com/kcl-lang/flask-demo-kcl-manifests.git
141+
ref:
142+
branch: main # Monitor the main branch of the configuration repository
143+
---
144+
apiVersion: krm.kcl.dev.fluxcd/v1alpha1
145+
kind: KCLRun
146+
metadata:
147+
name: kcl-git-controller
148+
namespace: default
149+
spec:
150+
sourceRef:
151+
kind: GitRepository
152+
name: kcl-deployment
153+
```
154+
155+
### 7. View Resources Using kubectl
156+
157+
You can use the following command to view the resources in the Kubernetes cluster
158+
159+
```shell
160+
kubectl get deplopments
161+
```
162+
163+
From the output, you can see that the deployed resources have been updated to the latest image
164+
165+
```shell
166+
NAME READY UP-TO-DATE AVAILABLE AGE
167+
flask-demo 1/1 1 1 16d
168+
```
169+
170+
## Conclusion
171+
172+
Through this article, we show how to use Flux KCL Controller and FluxCD to integrate KCL to create a GitOps automation pipeline, achieving efficient and stable construction of containerized applications. By using Flux KCL Controller and FluxCD to automatically update Docker image tags, and ensure that the configuration in Git is synchronized with the cluster state, we optimized the deployment process, achieved the separation of responsibilities between development and operations, and simplified the management of application configurations. This integration provides a transparent, traceable, and reproducible way to continuously deliver software, ensuring the flexibility of development and the stability of the production environment.

‎i18n/zh-CN/docusaurus-plugin-content-blog/2024-02-23-flux-kcl-controller/index.md‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
---
2+
slug: 2024-02-23-flux-kcl-controller
3+
title: flux-kcl-controller 助力 KCL & FluxCD 实现 GitOps
4+
authors:
5+
name: KCL Team
6+
title: KCL Team
7+
tags: [KCL, Github, FluxCD, GitOps]
8+
---
9+
10+
## 前言
11+
12+
在现代软件开发实践中，GitOps 已成为管理基础架构和应用程序的一种关键技术。能够显著提升自动化水平，减少人为干预，从而降低错误的发生率，并提高整体的操作效率。这种方法已在云原生等领域获得广泛应用。
13+
14+
在之前的文章中：我们分享了借助 ArgoCD，KCL 和 Github 实践 GitOps 的自动化流程。
15+
16+
- [使用 Github、Argo CD 和 KCL 实现 GitOps 以简化 DevOps](https://kcl-lang.io/zh-CN/blog/2023-07-31-kcl-github-argocd-gitops)
17+
18+
本文将继续扩展 KCL 在 GitOps 领域的实践，与持续集成工具 FluxCD 进行集成，并借助 KCL、GitHub 和 FluxCD 来提供一个具体的实践示例，详细阐述如何搭建和运行一个GitOps 自动化流程。
19+
20+
### 什么是 GitOps
21+
22+
GitOps 是一种基于 Git 的软件部署和操作模型，旨在利用 Git 的版本控制能力来管理和自动化基础设施及应用程序的部署。在 GitOps 中，Git 仓库不仅是代码的存储地，也是真实环境状态的反映。任何更改都通过对 Git 仓库的提交来实施，这些更改随后会被自动同步到生产环境中。
23+
24+
通过 GitOps, 可以有效
25+
- 增强开发和运维的协作：通过统一的 Git 工作流，开发人员和运维人员可以更协调地合作。
26+
- 提升部署的速度与安全性：GitOps通过自动化流程简化了部署，同时提供了必要的审计和回滚机制。
27+
- 提高系统的可追溯性：Git作为配置的单一来源，确保了每一次更改都被记录和追踪，便于监控和回顾。
28+
29+
### KCL & FluxCD
30+
31+
FluxCD 是一个实现 GitOps 模型的自动化工具，专门用于 Kubernetes 集群。它负责监控 Git 仓库中的变化，并确保 Kubernetes 集群的状态与仓库中定义的状态保持一致。FluxCD 的关键特点包括：
32+
33+
- 自动化同步：自动将Git仓库中的变更同步到 Kubernetes，实现配置的持续部署。
34+
- 声明式基础设施：通过声明式的配置文件来管理集群，使基础设施的版本控制更加直观。
35+
- 安全性与合规性：通过 Git 的分支和 PR 机制提供更安全的更改管理和审计跟踪。
36+
37+
KCL 通过其抽象和可编程特性能够显著简化复杂的 Kubernetes 部署配置。它将错误几率最小化，允许开发人员在编写阶段及时发现潜在问题，而非等到运行时。这意味着更少的配置模板和更强的多环境以及多租户配置能力，从而提升配置的可读性和可维护性。
38+
39+
利用 KCL，开发人员能够以声明式的手段精确定义所需的资源，结合 FluxCD，这种声明式的基础可以促进基础设施即代码（IaC）的实施，进而提高部署效率并且优化应用程序的配置管理。FluxCD 作为一个自动化的持续部署工具，加上对 KCL 的支持，为配置提供了一个易于管理的可视化界面。
40+
41+
在采用 GitOps 的流程中，开发人员和运维团队可以各司其职，通过修改应用和配置代码来管理应用程序的部署。GitOps 工具链，如 FluxCD，将自动同步这些更改，确保持续部署的同时保持环境状态的一致性。在部署过程中遇到任何问题，也可以便捷地利用 GitOps 工具链进行快速回滚，确保系统稳定性和业务连续性。
42+
43+
### Flux KCL Controller
44+
45+
Flux KCL Controller 是我们为 KCL 开发的一个 FluxCD Controller, 负责监控存储 KCL 程序的 Git 仓库。通过这个控制器，FluxCD 能够扩展其自动化部署的能力，实现对 KCL 语言编写的配置文件的持续监控和应用。
46+
47+
![flux-cd](/img/blog/2024-02-23-flux-kcl-controller/fluxcontroller.jpg)
48+
49+
借助 FluxCD 官方提供的 Source Controller ,Flux KCL Controller 可以定期检查与之关联的 Git 仓库中的 KCL 文件，一旦检测到仓库中有新的提交或更新，它便会自动触发配置的同步过程。这意味着，任何对 KCL 配置的更改都将被检测到，并自动反映到 Kubernetes 集群的状态中，从而维护配置的最新状态和一致性。
50+
51+
- [更多内容关于 Flux KCL Controller](https://github.com/kcl-lang/flux-kcl-controller)
52+
53+
## 案例介绍
54+
55+
我们仍然使用一个 Python Flask 应用和 Github Actions 作为 CI 示例，使用 Flux KCL Controller 来集成 FluxCD 的功能进行持续集成。
56+
57+
我们将 Python Flask 应用代码和配置代码分成两个仓库，以实现不同角色如开发人员和运维团队的关注点分离。
58+
59+
- 业务代码仓库: https://github.com/kcl-lang/flask-demo
60+
- 配置清单仓库: https://github.com/kcl-lang/flask-demo-kcl-manifests
61+
62+
整体工作流程如下：
63+
64+
![workflow](/img/blog/2024-02-23-flux-kcl-controller/workflow.jpg)
65+
66+
- 从 Github 拉取应用代码
67+
- 应用代码开发并提交到提交到 GitHub 存储库
68+
- 触发 GitHub Actions 对应用代码进行编译，生成容器镜像，并将容器镜像推送到 Docker Hub 容器注册表
69+
- 触发 GitHub Actions 根据 docker.io 容器注册表中容器镜像的版本号并同步更新 KCL 定义的 Kubernetes 清单部署文件
70+
- Flux KCL Controller 根据 Git 仓库的变更，获取 KCL 定义的 Kubernetes 清单更改并更新部署至 Kubernetes 集群。
71+
72+
## 具体步骤
73+
74+
### 0. 先决条件
75+
76+
- 熟悉 Unix/Linux 的基本命令
77+
- 熟悉 Git 以及 Github Action 使用
78+
- 了解 Kubernetes 基本知识
79+
- 了解 KCL 基本知识
80+
81+
### 1. 配置 Kubernetes 集群
82+
83+
- 安装 [K3d](https://github.com/k3d-io/k3d) 并创建一个集群
84+
85+
```bash
86+
k3d cluster create mycluster
87+
```
88+
89+
> 注意：你可以在此方案中使用其他方式创建您自己的 Kubernetes 集群，如 kind, minikube 等。
90+
91+
### 2. 安装和配置 Flux KCL Controller
92+
93+
- 通过以下命令在集群中安装 Flux KCL Controller
94+
95+
```bash
96+
git clone https://github.com/kcl-lang/flux-kcl-controller.git/ && cd flux-kcl-controller && make deploy
97+
```
98+
99+
更多详细内容关于 Flux-KCL-Controller 的安装和使用，请参考 [Flux-KCL-Controller](https://github.com/kcl-lang/flux-kcl-controller/blob/main/README-zh.md).
100+
101+
### 3. 获得业务代码
102+
103+
```shell
104+
git clone https://github.com/kcl-lang/flask-demo.git/
105+
cd flask-demo
106+
```
107+
108+
这是一个使用 Python 编写的 Web 应用，我们可以使用应用目录的 `Dockerfile` 来生成这个应用的容器镜像，同时可以通过 Github CI 自动构建 `flask_demo` 镜像。
109+
110+
在之前的文章中，我们已经完成了这部分的工作，这里就不再重复描述。[更多细节](https://kcl-lang.io/zh-CN/blog/2023-07-31-kcl-github-argocd-gitops/#3-%E8%8E%B7%E5%BE%97%E4%B8%9A%E5%8A%A1%E4%BB%A3%E7%A0%81)。
111+
112+
113+
### 4. 提交应用代码
114+
115+
flask-demo 仓库提交代码后，Github 会自动构建容器镜像，并将制品推送到 Docker hub 中，会再触发 flask-demo-kcl-manifests 仓库的 Action，[通过 KCL 自动化 API](/docs/user_docs/guides/automation) 修改部署清单仓库中的镜像地址。现在让我们为 flask-demo 仓库创建一个提交，我们可以看到代码提交后触发业务仓库 Github CI 流程
116+
117+
![](/img/docs/user_docs/guides/ci-integration/app-ci.png)
118+
119+
### 5. 配置自动更新
120+
121+
当业务仓库 Github CI 流程执行完成后，会自动在存放 KCL 资源配置的仓库触发一个 CI 自动更新配置并提交到 flask-demo-kcl-manifests main 分支。
122+
123+
在之前的文章中，我们已经完成了这部分的工作，因此在这里我们就不再重复，[更多细节](https://kcl-lang.io/zh-CN/blog/2023-07-31-kcl-github-argocd-gitops/#5-%E9%85%8D%E7%BD%AE%E8%87%AA%E5%8A%A8%E6%9B%B4%E6%96%B0)。
124+
125+
### 6. 使用 Flux KCL Controller 监控配置仓库
126+
127+
完成安装后，我们可以通过以下命令设置 Flux KCL Controller 监控配置仓库的地址，并根据配置内容自动更新 Kubernetes 集群中的资源。
128+
129+
```yaml
130+
apiVersion: source.toolkit.fluxcd.io/v1
131+
kind: GitRepository
132+
metadata:
133+
name: kcl-deployment
134+
namespace: default
135+
spec:
136+
interval: 10s # 每隔 10s 检查一次仓库
137+
url: https://github.com/kcl-lang/flask-demo-kcl-manifests.git
138+
ref:
139+
branch: main # 监控 main 分支
140+
---
141+
apiVersion: krm.kcl.dev.fluxcd/v1alpha1
142+
kind: KCLRun
143+
metadata:
144+
name: kcl-git-controller
145+
namespace: default
146+
spec:
147+
sourceRef:
148+
kind: GitRepository
149+
name: kcl-deployment
150+
```
151+
152+
### 7. 通过 kubectl 查看资源
153+
154+
通过以下命令，可以查看 Kubernetes 集群中的资源。
155+
156+
```shell
157+
kubectl get deplopments
158+
```
159+
160+
从输出中可以看到部署的资源已经更新为最新的镜像
161+
162+
```shell
163+
NAME READY UP-TO-DATE AVAILABLE AGE
164+
flask-demo 1/1 1 1 16d
165+
```
166+
167+
## 结论
168+
169+
通过本篇文章，我们探索了如何利用 Flux KCL Controller 和 FluxCD 集成 KCL 来创建 GitOps 自动化流水线，实现了对容器化应用的高效和稳定构建。结合使用 Flux KCL Controller 和 FluxCD 自动更新 Docker 镜像标签，并确保 Git 中的配置与集群状态保持同步，优化了部署流程，实现了开发和运维之间的职责分离，并简化了应用程序配置的管理过程。这个集成方案为我们提供了一种透明、可追踪和可重现的方式来持续交付软件，确保了开发的灵活性和生产环境的稳定性。

‎static/img/blog/2024-02-23-flux-kcl-controller/fluxcontroller.png‎

142 KB
Loading

‎static/img/blog/2024-02-23-flux-kcl-controller/workflow.png‎

444 KB
Loading

0 commit comments

Comments
 (0)