自动化部署使用文档
0. 自动化部署设计
1. 资源包准备
编辑 ansible_root/images/imom_release.env, ansible_root/images/mbm_release.env 文件, 确认imom, mbm 要发版的镜像列表。
cat ansible_root/images/imom_release.env
export IMAGE_PULL_SECRETS="default-secret"
export IMAGE_BASE="192.168.1.104:5000"
# biz images
export GATEWAY_APP_IMAGE_NAME="${IMAGE_BASE}/private/admin-gateway:v0.0.42"
export MPM_APP_IMAGE_NAME="${IMAGE_BASE}/private/smt-mes-biz:20250815.1-release"
export KERNEL_APP_IMAGE_NAME="${IMAGE_BASE}/private/kernel-biz:v0.0.138"
export LED_MES_APP_IMAGE_NAME="${IMAGE_BASE}/private/led-mes-biz:v0.0.321"
export MDM_APP_IMAGE_NAME="${IMAGE_BASE}/private/mdm-biz:v0.0-dev46"
export MES_REPORT_APP_IMAGE_NAME="${IMAGE_BASE}/private/mom-report-biz:v0.0.22-sit"
export TOOLS_APP_IMAGE_NAME="${IMAGE_BASE}/private/em-tools-biz:20250815.1-release"
export EAM_APP_IMAGE_NAME="${IMAGE_BASE}/private/mom-eam-biz:20250815.1-release"
export FLOW_APP_IMAGE_NAME="${IMAGE_BASE}/private/imom-flow-biz:v0.0-dev113"
export PUBLIC_APP_IMAGE_NAME="${IMAGE_BASE}/private/mom-public-biz:20250815.1-release"
export QMS_FIM_APP_IMAGE_NAME="${IMAGE_BASE}/private/fim-biz:v0.0.21"
export QMS_PROCESS_CONTROL_APP_IMAGE_NAME="${IMAGE_BASE}/private/process-control-biz:v0.0.460"
export QMS_SPC_APP_IMAGE_NAME="${IMAGE_BASE}/private/qms-spc-biz:v0.0-dev66"
export QMS_SPC_CFG_APP_IMAGE_NAME="${IMAGE_BASE}/private/qms-spc-cfg-biz:v0.0-dev126"
export SMT_APP_IMAGE_NAME="${IMAGE_BASE}/private/smt-mes-biz:20250815.1-release"
export LES_APP_IMAGE_NAME="${IMAGE_BASE}/imom-release/imom-les:20250804-release"
# ui images
export LES_UI_APP_IMAGE_NAME="${IMAGE_BASE}/imom/imom-les-ui-master:20250728.146"
export MES_UI_APP_IMAGE_NAME="${IMAGE_BASE}/imom/imom-mes-ui:2025.815.0-beta.3"
export MPM_UI_APP_IMAGE_NAME="${IMAGE_BASE}/imom/imom-mpm-ui-dev:20250812.282"
export PUBLIC_UI_APP_IMAGE_NAME="${IMAGE_BASE}/imom/imom-public-ui-dev:20250815.320"
export QMS_UI_APP_IMAGE_NAME="${IMAGE_BASE}/imom/imom-qms-ui-dev:20250815.318"
cat ansible_root/images/mbm_release.env
export IMAGE_PULL_SECRETS="default-secret"
export IMAGE_BASE="192.168.1.104:5000"
# biz images
export GATEWAY_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/api-gateway-service:2d02c75_2025.0526.1032.19_dev-x86_64"
export MDM_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/mdm-service:8d1bc2f_2025.0808.1122.31_dev-x86_64"
export QM_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/qm-service:07ec2d1_2025.0530.1523.21_dev-x86_64"
export LABEL_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/label-service:cca8f0d_2025.0609.1553.08_dev-x86_64"
export MSM_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/msm-service:448e5a3_2025.0714.1649.04_dev-x86_64"
export EM_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/em-service:40da4d4_2025.0625.1728.24_dev-x86_64"
export LES_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/les-service:bb61b08_2025.0808.1616.12_dev-x86_64"
export SFC_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/sfc-service:26eab7b_2025.0808.1433.40_dev-x86_64"
export WOM_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/wom-service:60cce36_2025.0806.1440.01_dev-x86_64"
# ui images
export MOM_WEB_APP_IMAGE_NAME="${IMAGE_BASE}/ipdc-mbm/mom-web:dev_20250808153220_b621287-x86_64"
export ORCHE_WEB_APP_IMAGE_NAME="${IMAGE_BASE}/mom/web-framework:dev_20250519110325_8e90ce9-x86_64"
修改好后执行 save_imagse.sh 脚本打包镜像
bash -x save_imagse.sh
执行完毕后，可确认docker 镜像已放置在 ansible_root/images 内:
[root@ansible ~]# ls ansible_root/images
centos7-ansible_latest.tar ipdc-mbm_msm-service_448e5a3_2025.0714.1649.04_dev-x86_64.tar private_mom-public-biz_20250815.1-release.tar
imom_imom-les-ui-master_20250728.146.tar ipdc-mbm_wom-service_60cce36_2025.0806.1440.01_dev-x86_64.tar private_mom-report-biz_v0.0.22-sit.tar
imom_imom-mes-ui_2025.815.0-beta.3.tar mbm_release.env private_process-control-biz_v0.0.460.tar
imom_imom-mpm-ui-dev_20250812.282.tar private_admin-gateway_v0.0.42.tar private_qms-spc-biz_v0.0-dev66.tar
imom_imom-public-ui-dev_20250815.320.tar private_em-tools-biz_20250815.1-release.tar private_qms-spc-cfg-biz_v0.0-dev126.tar
imom_imom-qms-ui-dev_20250815.318.tar private_fim-biz_v0.0.21.tar private_smt-mes-biz_20250815.1-release.tar
imom_release.env private_imom-flow-biz_v0.0-dev113.tar redis_8.0.2-debian-12-r4.tar
imom-release_imom-les_20250804-release.tar private_kernel-biz_v0.0.138.tar registry.tar
ipdc-mbm_les-service_bb61b08_2025.0808.1616.12_dev-x86_64.tar private_led-mes-biz_v0.0.321.tar save_image.sh
ipdc-mbm_mdm-service_8d1bc2f_2025.0808.1122.31_dev-x86_64.tar private_mdm-biz_v0.0-dev46.tar
ipdc-mbm_mom-web_dev_20250808153220_b621287-x86_64.tar private_mom-eam-biz_20250815.1-release.tar
2. 部署规划
先对所有的部署机器安装 centos7.9 最小化系统，并设置好网络ip。保证以下的机器在同一个局域网内:
ansible部署机 x1 (代号:ansible)
kk部署机 x1 (代号:kk)
nfs x1 (代号:broker_nfs)
mysql xN (代号:broker_mysql) 如果是单台服务器, 则是单机; 否则第一台是主节点，之后是从节点。
mysql_hw xN (代号:broker_mysql_hw) 如果是单台服务器, 则是单机; 否则第一台是主节点，之后是从节点。
docker镜像仓库x1 (代号:broker_docker_registry)
k8s主节点xN(代号: master) 要求为奇数, 至少一台
k8s工作节点xN(代号: node) 一台或以上
推荐 ansible部署机和kk部署机 使用同一台服务器。
2.1 配置主机以及角色
根据上文的部署规划, 编辑 ansible_root/host.ini 该文件。
cat host.ini
# hosts.ini
[host_broker_nfs]
192.168.1.101
[host_broker_mysql]
192.168.1.102
192.168.1.109
[host_broker_mysql_hw]
192.168.1.103
192.168.1.110
[host_broker_docker_registry]
192.168.1.104
[host_kk]
192.168.1.104
[k8s_master]
192.168.1.111 ansible_user=root ansible_password=123456
[k8s_node]
192.168.1.113 ansible_user=root ansible_password=123456
192.168.1.114 ansible_user=root ansible_password=123456
# 仅用于部署日志监控
[host_monitor]
192.168.1.99
192.168.1.116
192.168.1.112
192.168.1.115
[all:vars]
ansible_user=root
ansible_ssh_pass=123456
2.2 配置主机以及角色
编辑 ansible_root/vars/host.yml 文件，设置部署机器上安装的中间件软件信息
cat vars/host.yml
# host 主机系统 centos79 或 rocky97
os: "rocky97"
# 主机环境 -> kubesphere部署完成
mysql_user: "root"
# mysql 的密码不能太简单
mysql_password: "nhdTaVMSAC"
docker_registry_address: "192.168.1.104:5000"
nfs_address: "192.168.1.101"
nexus3_address: "192.168.1.105:8081"
kube_cluster_name: "sie-kubesphere"
编辑 ansible_root/vars/k8s_app.yml 文件，设置k8s集群上安装的中间件，业务应用的软件信息
cat vars/k8s_app.yml
# host 主机系统 centos79 或 rocky97
os: "rocky97"
# k8s app, 中间件安装环境配置
imom_namespace: "imom"
imom_common_namespace: "common"
imom_elk_namespace: "elk"
imom_disable_elk: "1"
hw_namespace: "mbm"
image_pull_secrets: "default-secret"
default_secrets_content: "eyJhdXRocyI6eyJzd3IuY24tc291dGgtNC5teWh1YXdlaWNsb3VkLmNvbSI6eyJhdXRoIjoiWTI0dGMyOTFkR2d0TkVCSVUxUXpRVEJQTjFKVE16YzBNVEEzU0RkRU1EbzFZamczWXpNd01EWmxaVGc1TkRGbE5UQTBNVEl3WW1Ka01EUTVORE5sTmpJMFpURTJaREEwTmpWa05tRmhOMkkzTlRGa1l6aGpabUk0Tmpaa05UYzIifSwic3dyLmNuLXNvdXRoLTQubXlodWF3ZWljbG91ZC5jb206NDQzIjp7ImF1dGgiOiJZMjR0YzI5MWRHZ3RORUJJVTFRelFUQlBOMUpUTXpjME1UQTNTRGRFTURvMVlqZzNZek13TURabFpUZzVOREZsTlRBME1USXdZbUprTURRNU5ETmxOakkwWlRFMlpEQTBOalZrTm1GaE4ySTNOVEZrWXpoalptSTROalprTlRjMiJ9fX0="
default_storage_class: "nfs-client"
external_base_url: "http://192.168.1.112:32555"
imom_gateway_host: "192.168.1.112:32556"
enable_imom_apps: "devops-biz,imom-aps,imom-eam,imom-flow-biz,imom-gateway,imom-kernel,imom-les,imom-mdm,imom-mes-public,imom-mpm,imom-process-control,imom-smt,imom-spc-config,imom-spc,imom-tools,imom-aps-ui,imom-les-ui,imom-mes-ui,imom-mpm-ui,imom-public-ui,imom-qms-ui"
docker_registry_address: "192.168.1.104:5000"
nfs_address: "192.168.1.101"
nexus3_address: "192.168.1.105:8081"
mysql_address: "192.168.1.116"
mysql_user: "root"
mysql_password: "nhdTaVMSAC"
redis_address: "sie-redis-broker.{{ imom_common_namespace }}"
redis_master_address: "sie-redis-sentinel-master-svc.{{ imom_common_namespace }}"
redis_password: "Dme123456"
redis_mode: "SENTINEL"
minio_address: "sie-minio-broker.{{ imom_common_namespace }}"
minio_user: "admin"
minio_password: "Minio123456"
powerjob_address: "powerjob-svc.{{ imom_common_namespace }}"
# idme
#idme_address: "http://192.168.168.207:30083"
idme_address: "http://192.168.1.99:30083"
idme_sub_app_id: "rdm_e3231ab62e9d4771bd0df793f62d2282_app"
idme_app_id: "e3231ab62e9d4771bd0df793f62d2282"
idme_mysql_address: "192.168.181.153"
idme_mysql_user: "root"
idme_mysql_password: "nhdTaVMSAC"
idme_deploy_version: "2.25.060.8.20250626.6"
# hw mbm
hw_mbm_address: "http://mbm-api-gateway.{{ hw_namespace }}:9090"
# ----------- huawei config -------- #
enable_mbm_apps: "api-gateway-service,em-service,label-service,les-service,mdm-service,mbm-mom-web,msm-service,mbm-orche-web,qm-service,sfc-service,wom-service,mpdm-service"
# 管理员账号 admin Ghgf@1234
hw_tenant_id: "067cfa58c8404a908780bda7e934e1b1"
hw_org_name: "贵州贵航红阳机械"
hw_redis_address: "redis-cluster-service.{{ hw_namespace }}"
hw_redis_password: "Rdis123456"
hw_redis_mode: "CLUSTER"
hw_mysql_address: "192.168.1.116"
hw_mysql_user: "root"
hw_mysql_password: "nhdTaVMSAC"
hw_minio_address: "sie-minio-broker.{{ imom_common_namespace }}"
hw_minio_user: "admin"
hw_minio_password: "Minio123456"
# 修复华为qm flyway bug
hw_gen_qm_xdm_db: "1"
# ----------- node affinity config(common和mbm为核心应用) -------- #
affinity_enable: "0"
affinity_common_node: "node1"
affinity_mbm_node: "node2"
affinity_imom_node: "node3"
# 可为: NoSchedule, PreferNoSchedule 建议: PreferNoSchedule
affinity_taint_policy: "PreferNoSchedule"
2.3 部署包上传
将 ansible_root 该文件夹分别上传到 ansible, kk 部署机器的 /root 目录下(如果ansible, kk使用的是同一台机器，即传送到ansible这台部署机器即可)
[root@ansiblerun ~]# pwd
/root
[root@ansiblerun ~]# ls
anaconda-ks.cfg ansible_root
2. 自动化部署
进入 /root/ansible_root 目录, 确认images 目录存在 centos7-ansible_latest.tar 文件， 然后执行:
# 设置脚本可被执行
chmod 755 auto.sh
# 开始自动化部署
./auto.sh
耐心等待kubesphere 集群, 以及相关的imom, mbm 应用部署完成。
部署完毕后， 使用以下的信息访问:
kubesphere 管理台: http://{MASTER节点IP}:30880 用户: admin 密码: P@88w0rd
华为mbm前端: http://{MASTER节点IP}:32555用户: admin 密码: Ghgf@1234
3. 部署监控
现提供自动化部署的日志监控工具查看ansible 执行过程中， 所有的部署机器shell 脚本的执行日志，使用方法如下:
cd /root/ansible_root
chmod 755 monitor.sh
# 查看shell脚本日志, 并写入到logs 目录
./monitor.sh
如果部署过程中遇到错误, 提供 logs 目录下的所有文件便于分析。
4. 执行细分任务(高级)
先自动化部署提供了"细分任务" 的执行功能，可指定单独执行某个小任务，而无需完整重跑自动化执行脚本。现细分任务包括:
4.1 按阶段执行任务:
仅执行第一阶段: 从部署机器环境安装到kubesphere安装完成
./auto.sh host
仅执行第二阶段: 在kubesphere 上安装imom, 华为mbm 应用以及其中间件
./auto.sh k8s_app
4.2 按主任务执行任务:
执行host主任务
./auto.sh host all
执行host_broker主任务
./auto.sh host_broker all
执行k8s主任务
./auto.sh k8s all
执行k8s_app_broker主任务
./auto.sh k8s_app_broker all
执行k8s_app主任务
./auto.sh k8s_app all
执行k8s_hw_app_broker主任务
./auto.sh k8s_hw_app_broker all
执行k8s_hw_app主任务
./auto.sh k8s_hw_app all
4.3 按主任务下的子任务来执行任务:
执行host主任务
./auto.sh host all
执行host_broker主任务下的子任务(docker|docker_registry|mysql|nfs)
./auto.sh host_broker docker|docker_registry|mysql|nfs
执行k8s主任务下的子任务(image|image_with_kk|k8s_with_kk)
./auto.sh k8s yum|docker|config|kk_image|docker_image|kubesphere
执行k8s_app_broker主任务下的子任务(minio|nfs_client|powerjob|redis)
./auto.sh k8s_app_broker minio|nfs_client|powerjob|redis|kkfileview|elk|idme
执行k8s_hw_app_broker主任务下的子任务(redis)
./auto.sh k8s_hw_app_broker redis
执行k8s_affinity主任务下的子任务(affinity)
./auto.sh k8s_affinity affinity