基于Kubernetes/Jenkins/Harbor/Sonarqube搭建微服务CICD系统

**项目说明**

14个JAVA spring cloud 微服务项目

5个Node项目

1个Node项目VUE生成静态页面做前端+nginx

pgsql

redis

rabbitmq

emqx

项目思维导图

**构建流程：**

一、项目软件搭建

1，K8S搭建（参考阿良部署文档：kubeadmin安装单主K8S集群）

2，jenkins搭建

https://jenkins.io/doc/book/installing/#fedora

3，Harbor搭建

https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md

4，Sonarqube搭建

sonarqube检测java项目：https://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+Maven

sonarqube检测node项目：https://www.npmjs.com/package/sonarqube-scanner

二、CICD系统中使用的资料

1，jenkins pipeline（JAVA服务的）

node?{
???def?mvnHome?=?tool?'Maven'
???
???REGISTRY?=?'hb.shubing.vip/k12soft'
???//environment?{?
???//REGISTRY?=?'hb.shubing.vip/k12soft'
???//}
???
???
???stage('Pull?Code?from?Gitlab.')?{
????????git?branch:?'test_4',?credentialsId:?'1',?url:?'http://10.0.0.82:9999/k12soft/xxx.git'
???}
???stage('Maven?Package?and?Sonar.')?{
????????withMaven(jdk:'JDK'){
???????//?def??JAVA_HOME?=?"/usr/lib/jdk1.8.0_121"
????????sh?"'${mvnHome}/bin/mvn'?clean?package?-e?-U?-Dmaven.test.skip=true?sonar:sonar"
????????//sh?"'${mvnHome}/bin/mvn'?-f?ebase-api/pom.xml?clean?package?sonar:sonar"
????????}
????}
????
???stage('Build?Docker?Image.')?{
????????sh?"sudo?docker?build?-t?$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}?."
????}
????stage('Push?Docker?Image?to?Harbor.')?{
????????sh?"sudo?docker?push?$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}"
????}
????stage('Deploy?K8S?Service.')?{
????????sh?"sudo?ssh?'root@10.0.0.102'?'cd?/root/xtcloud?&&?sh?apply-config.sh?${BUILD_ID}?${JOB_BASE_NAME}'"
????????dingTalk?accessToken:?'https://oapi.dingtalk.com/robot/send?access_token=8241e530e0f2e68f74da6b78f893e2f526db4086a88e76de7348ddda101f13xx',?imageUrl:?'',?jenkinsUrl:?'',?message:?"??构建&?K8S发布成功！",?notifyPeople:?''
????}
????
}

2，jenkins pipeline（Node服务的）

node?{
???
???REGISTRY?=?'hb.shubing.vip/k12soft'
???stage('Pull?Code?from?Gitlab.')?{
????????git?branch:?'zangyunji',?credentialsId:?'1',?url:?'http://10.0.0.82:9999/k12soft/xxx.git'
???}
???//stage('Node?Install?Modules.')?{
?????//???sh?"source?/etc/profile.d/node.sh?&&?npm?i"
????//}
????stage('Node?Build?and?Sonar.')?{
????????sh?"source?/etc/profile.d/node.sh?&&?sonar-scanner"
????????sh?"source?/etc/profile.d/node.sh?&&?npm?run?tsc"
????}
????????
???stage('Build?Docker?Image.')?{
????????sh?"sudo?docker?build?-t?$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}?."
????}
????stage('Push?Docker?Image?to?Harbor.')?{
????????sh?"sudo?docker?push?$REGISTRY/${JOB_BASE_NAME}:${BUILD_ID}"
????}
????stage('Deploy?K8S?Service.')?{
????????sh?"sudo?ssh?'root@10.0.0.102'?'cd?/root/xtcloud?&&?sh?apply-config.sh?${BUILD_ID}?${JOB_BASE_NAME}'"
????????dingTalk?accessToken:?'https://oapi.dingtalk.com/robot/send?access_token=8241e530e0f2e68f74da6b78f893e2f526db4086a88e76de7348ddda101f13xx',?imageUrl:?'',?jenkinsUrl:?'',?message:?"??构建&?K8S发布成功！",?notifyPeople:?''
????}
????
????
}

3，dockerfile （JAVA服务的）

FROM?java:8u111
MAINTAINER?gongshubing@weds.com.cn
#?定义变量
ENV?WORK_DIR?/opt
ENV?LOG_DIR?/data/logs
EXPOSE?8086
#?从主机?copy?到容器里面
COPY?target/bepf_archives.jar?$WORK_DIR
WORKDIR?$WORK_DIR
ENTRYPOINT?["sh",?"-c"]
CMD?["java?-server?-Dspring.profiles.active=ops?-Xmx512m?-Xms256m?-Xmn256m?-XX:+UseG1GC?-XX:+DisableExplicitGC?-Duser.timezone=GMT+8?-jar?*.jar?>?/dev/null"
]

4，dockerfile（NODE服务的）

FROM?node:10.15.3
MAINTAINER?gongshubing@weds.com.cn
#?定义变量
ENV?WORK_DIR?/opt
ENV?LOG_DIR?/data/logs
ENV?EGG_SERVER_ENV?ops
EXPOSE?8086
#?从主机?copy?到容器里面
COPY?./?$WORK_DIR/
RUN?sed?-i?'s/\-\-daemon//g'?$WORK_DIR/package.json
WORKDIR?$WORK_DIR
ENTRYPOINT?["sh",?"-c"]
CMD?["npm?start"]

5，K8S 编排yaml文件（所有项目编排文件一样，不同之处只有镜像URL/IP/PORT）：pod中的日志文件写入nfs

#?deployment
---
apiVersion:?apps/v1
kind:?Deployment
metadata:
??name:?xtcloud-appgateway-deployment
??labels:
????app:?xtcloud-appgateway
spec:
??replicas:?1
??selector:
????matchLabels:
??????app:?xtcloud-appgateway
??template:
????metadata:
??????labels:
????????app:?xtcloud-appgateway
????spec:
??????containers:
??????-?name:?xtcloud-appgateway
????????image:?hb.shubing.vip/k12soft/xtcloud-appgateway:latest
????????ports:
????????-?containerPort:?8097
????????volumeMounts:
??????????-?name:?xtcloud-logs-persistent-storage
????????????mountPath:?/data/logs
??????volumes:
??????-?name:?xtcloud-logs-persistent-storage
????????nfs:
??????????path:?/nfs
??????????server:?10.0.60.67
??????imagePullSecrets:
????????-?name:?weds-harbor-secret
#?service
---
kind:?Service
apiVersion:?v1
metadata:
??name:?xtcloud-appgateway-service
spec:
??clusterIP:?192.168.1.15
??selector:
????app:?xtcloud-appgateway
??ports:
????-?protocol:?TCP
??????port:?8097
??????targetPort:?8097

6，jenkins调用编排K8s脚本

#!/bin/bash

TAG=$1
JOB_NAME=$2

#sed?"s/latest/$TAG/"?${JOB_NAME}.yml?|?kubectl?apply?-f?-
sed?"s/latest/$TAG/"?${JOB_NAME}.yml?|?kubectl?replace?--force?-f?-

二、效果图