自有测试平台LunarLink部署生产环境实践

geekbing2024-03-02自动化测试测试平台

原理

利用 Jenkins 拉取 GitLab 仓库代码 --> 构建 Docker 镜像 --> 上传 Docker 镜像到自建的 Harbor 镜像仓库 --> 利用 sh 脚本生成 docker-compose.yml --> 将 docker-compose.yml 推送到部署服务器 --> 部署服务器再执行docker-compose up -d命令部署

准备工作

3 台服务器，1 台安装 Docker + Jenkins + Git + Harbor 做持续集成，参照open in new window
1 台部署前端服务（Vue2、Nginx）、后端服务（Django 、定时/异步任务、录制流量代理），需要先安装 Docker
1 台部署 MySQL 数据库&&中间件 RabbitMQ、Redis，需要先安装 Dockeropen in new window

TIP

如果平台访问人数不多，服务器配置足够，可以将 3 台服务器缩减至 1 台，所有这些操作均在 1 台服务器上面执行

操作步骤

Docker

根目录下的build_and_push.sh脚本构建 Docker 镜像并推送到 Harbor 镜像仓库

#!/bin/bash
DOMAIN=$1
VERSION="1.0.0"

# Build the Docker image
docker build -t lunar-link-"$DOMAIN" -f ./deployment/"$DOMAIN"/Dockerfile .

# Tag the Docker image
docker tag lunar-link-"$DOMAIN" ip:port/lunar_link/"$DOMAIN":"$VERSION" # ip:port 为 Harbor 镜像仓库地址

# Login to the Docker registry
docker login -u 账号 -p 密码 ip:port  # 登录镜像仓库

# Push the Docker image
docker push ip:port/lunar_link/"$DOMAIN":"$VERSION"  # 推送到镜像仓库

根目录下的create_compose_file.sh创建 docker-compose.yml 文件脚本

#!/bin/bash

DJANGO_IMAGE_TAG="1.0.0"
WEB_IMAGE_TAG="1.0.0"
CELERY_IMAGE_TAG="1.0.0"
PROXY_IMAGE_TAG="1.0.0"

cat > docker-compose.yml <<EOF
version: "3"
services:
  lunar-link-django:
    image: ip:port/lunar_link/django:${DJANGO_IMAGE_TAG}  # ip:port 为 Harbor 镜像仓库地址
    container_name: lunar-link-django
    working_dir: /backend
    environment:
      PYTHONUNBUFFERED: 1
      TZ: Asia/Shanghai
    expose:
      - 8000
    restart: always

  lunar-link-web:
    image: ip:port/lunar_link/web:${WEB_IMAGE_TAG}
    container_name: lunar-link-web
    depends_on:
      - lunar-link-django
    ports:
      - "8081:8081"
    expose:
      - 8081
    environment:
      TZ: Asia/Shanghai
    restart: always

  lunar-link-celery:
    image: ip:port/lunar_link/celery:${CELERY_IMAGE_TAG}
    container_name: lunar-link-celery
    depends_on:
      - lunar-link-django
    environment:
      PYTHONUNBUFFERED: 1
      TZ: Asia/Shanghai
    restart: always

  lunar-link-proxy:
    image: ip:port/lunar_link/proxy:${PROXY_IMAGE_TAG}
    container_name: lunar-link-proxy
    depends_on:
      - lunar-link-django
    environment:
      PYTHONUNBUFFERED: 1
      TZ: Asia/Shanghai
    ports:
      - "7778:7778"
    expose:
      - 7778
    restart: always
EOF

Jenkins

安装 Publish over SSH 插件，用于连接部署服务器并推送 docker-compose.yml 文件

在 Jenkins 所在的机器上生成秘钥，一路按下 Enter 采用默认值

# 生成秘钥，一路按回车键
ssh-keygen

# 查看公钥，验证是否创建成功
cat ~/.ssh/id_rsa.pub

将 Jenkins 所在服务器的公钥拷贝到远服务器，如当前远程服务器的 ip 为 192.168.3.84
```
ssh-copy-id 192.168.3.84
```
进入 Manage Jenkins --> System 拉到底部的Publish over SSH 区域填入 Jenkins 所在服务器的私钥
```
# 查看私钥
cat id_rsa
```
配置 SSH Servers 连接远程服务器
选择 New Item 创建 Pipeline ，名称为 LunarLink。
进入Configure，勾选 This project is parameterized，选择 Boolean Parameter
进入Pipeline Syntax 生成连接远程服务器的Pipeline Script

配置Pipeline Script。

deploy9

// Pipeline script 详细代码如下
pipeline {
    agent {
        label 'slave3'  // Jenkins 节点 slave3 执行构建任务
    }
    
    parameters {
        booleanParam(defaultValue: true, description: 'Do you want to deploy lunar-link-proxy?', name: 'DEPLOY_PROXY')
    }

    stages {
        stage('从gitlab中拉取代码') {
            steps {
                git credentialsId: 'git凭证', url: '代码仓库地址'  // 代码仓库地址为http开头
            }
        }
        stage('打包构建docker镜像&&镜像打标签并上传') {
            steps {
                sh 'chmod +x ./build_and_push.sh'
                script {
                    if (params.DEPLOY_PROXY) {
                        sh './build_and_push.sh proxy'  // 根据 DEPLOY_PROXY 变量判断是否构建代理Docker镜像
                    }
                }
                sh './build_and_push.sh web'
                sh './build_and_push.sh django'
                sh './build_and_push.sh celery'
            }
        }
        stage('创建docker-compose.yml') {
            steps {
                sh 'chmod +x ./create_compose_file.sh'
                sh './create_compose_file.sh'
            }
        }  
        stage('远程部署应用') {
            steps {
                script {
                    def deployActions = ""

                    if (params.DEPLOY_PROXY) {  
                        deployActions = "sudo docker-compose down && sudo docker-compose pull && sudo docker-compose up -d"
                    } else {
                        deployActions = "sudo docker-compose stop lunar-link-django lunar-link-web lunar-link-celery && " +
                                        "sudo docker-compose pull lunar-link-django lunar-link-web lunar-link-celery && " +
                                        "sudo docker-compose up -d lunar-link-django lunar-link-web lunar-link-celery"
                    }

                    // 命令组合
                    def execCommand = "cd /home/jdkapp/tester && ${deployActions}"
                   // 推送 docker-compose.yml 到部署服务器并执行 execCommand 命令
                    sshPublisher(publishers: [sshPublisherDesc(configName: 'product_server',
                    transfers: [sshTransfer(cleanRemote: false, excludes: '',
                    execCommand: execCommand,
                    execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false,
                    patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false,
                    removePrefix: '', sourceFiles: 'docker-compose.yml')], usePromotionTimestamp: false,
                    useWorkspaceInPromotion: false, verbose: false)])
                }
            }
        }
        stage('企业微信通知') {
            steps {
                script {
                    wrap([$class: 'BuildUser']){
                        echo "full name is $BUILD_USER"
                        echo "user id is $BUILD_USER_ID"
                        echo "user email is ${env.BUILD_USER_EMAIL}"
                        env.BUILD_USER = "${BUILD_USER}"
                    }
                }                
            }            
        }  
    }
    post {  // 构建成功失败均发送企微同志
        success {
            script {
                WeiXinSuccess()
            }            
        }
        failure {
            script {
                WeiXinFailure()
            }
        }
        unstable{
            script {
                WeiXinSuccess()
            }             
        }
    }
}

def WeiXinSuccess(){
        sh """
            curl --location --request POST 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=企微机器人Webhook地址' \
                --header 'Content-Type: application/json' \
                --data '{
                    "msgtype": "markdown",
                    "markdown": {
                        "content": "<font color=\'warning\'>**Jenkins任务通知**</font> \n
                        >构建人: <font color=\'comment\'>${env.BUILD_USER}</font>
                        >构建时间: <font color=\'comment\'>${BUILD_TIMESTAMP}</font>
                        >运行时长: <font color=\'comment\'>${currentBuild.durationString}</font>
                        >任务名称: <font color=\'comment\'>${JOB_BASE_NAME}</font>
                        >构建日志: <font color=\'comment\'>[点击查看](${BUILD_URL}/console)</font>
                        >构建状态: <font color=\'info\'>**Success**</font>"                                     
                    }
                }'
        """
}

def WeiXinFailure(){
        sh """
            curl --location --request POST 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=企微机器人Webhook地址' \
                --header 'Content-Type: application/json' \
                --data '{
                    "msgtype": "markdown",
                    "markdown": {
                        "content": "<font color=\'warning\'>**Jenkins任务通知**</font> \n
                        >构建人: <font color=\'comment\'>${env.BUILD_USER}</font>
                        >构建时间: <font color=\'comment\'>${BUILD_TIMESTAMP}</font>
                        >运行时长: <font color=\'comment\'>${currentBuild.durationString}</font>
                        >任务名称: <font color=\'comment\'>${JOB_BASE_NAME}</font>
                        >构建日志: <font color=\'comment\'>[点击查看](${BUILD_URL}/console)</font>
                        >构建状态: <font color=\'comment\'>**Failure**</font>"      
                    }
                }'
        """
}

Nginx

LunarLink/deployment/web/nginx.conf

server {
    listen 8081;
    server_name 192.168.3.84;  # 替换成要部署的服务器ip
    client_max_body_size 100M;

    # 开启gzip
    gzip on;
    # https://blog.csdn.net/fxss5201/article/details/106535475
    gzip_static on;
    gzip_proxied any;
    # 低于1kb的资源不压缩
    gzip_min_length 1k;
    gzip_buffers 4 16k;
    gzip_comp_level 6;
    # 需要压缩的类型
    gzip_types text/plain application/javascript text/css application/xml text/javascript application/json;
    # 是否添加“Vary: Accept-Encoding”响应头
    gzip_vary on;

    location /django_static/ {
        alias /www/LunarLink/static/;
    }

    location /api/ {
      # 接口转发
        proxy_pass http://lunar-link-django:8000/api/;
        proxy_set_header Host $host;
        proxy_set_header Cookie $http_cookie;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_redirect default;
        add_header Access-Control-Allow-Origin *;
        add_header Access-Control-Allow-Headers X-Requested-With;
        add_header Access-Control-Allow-Methods GET,POST,OPTIONS;
    }

    location /admin/ {
        proxy_pass http://lunar-link-django:8000/admin/;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }

    location / {
        root /usr/share/nginx/html;
        index  index.html index.htm;
        try_files $uri $uri/ /index.html;
    }
}

前端

LunarLink/frontend/config/prod.env.js

"use strict";
// 生产环境的环境变量配置
const LunarLink = process.env.LUNAR_LINK || "LunarLink";
module.exports = {
    NODE_ENV: '"production"',
    DOCS_URL: '"http://192.168.3.84:8888/docs/guide/introduce.html"',  // 测试平台指南
    LUNAR_LINK: "'" + LunarLink + "'",
    RECORD_CASE_DOCS_URL:
        '"http://192.168.3.84:8888/docs/guide/test_case.html"',  // 测试平台操作手册
    VUE_APP_BASE_URL: '""'  // 接口地址 Nginx 会自动转发到后端，保持为空即可
};

后端

LunarLink/backend/conf/docker.py

# -*- coding: utf-8 -*-
"""
@File    : docker.py
@Time    : 
@Author  : 
"""
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

# ================================================= #
# ************** mysql数据库 配置  ************** #
# ================================================= #
# 数据库地址
DATABASE_HOST = ""
# 数据库端口
DATABASE_PORT = 3308
# 数据库用户名
DATABASE_USER = ""
# 数据库密码
DATABASE_PASSWORD = ""
# 数据库名
DATABASE_NAME = "lunar_prod"

# ================================================= #
# ************** RabbitMQ配置 ************** #
# ================================================= #
MQ_USER = "guest"
MQ_PASSWORD = ""
MQ_HOST = ""
MQ_URL = f"amqp://{MQ_USER}:{MQ_PASSWORD}@{MQ_HOST}:5672//"

# ================================================= #
# ************** Redis配置 ************** #
# ================================================= #
REDIS_ON = True
REDIS_HOST = ""
REDIS_PASSWORD = ""
REDIS_PORT = 6379
REDIS_DB = 0

# ================================================= #
# ************** Sentry监控配置  ************** #
# ================================================= #
sentry_sdk.init(
    dsn="",  # Sentry监控DSN，没有可不填
    integrations=[DjangoIntegration()],
    # Set traces_sample_rate to 1.0 to capture 100%
    # of transactions for performance monitoring.
    # We recommend adjusting this value in production.
    traces_sample_rate=1.0,
    # If you wish to associate users to errors (assuming you are using
    # django.contrib.auth) you may enable sending PII data.
    send_default_pii=True,
)

# ================================================= #
# ************** 其他 配置  ************** #
# ================================================= #
DEBUG = False  # 线上环境请设置为False
# 启动登录日志记录(通过调用api获取ip详细地址。如果是内网，关闭即可)
ENABLE_LOGIN_ANALYSIS_LOG = True
ALLOWED_HOSTS = ["*"]
BASE_REPORT_URL = "http://47.119.28.171:8081/api/lunarlink/reports"  # 替换成部署的服务器地址或域名
IM_REPORT_SETTING = {
    "base_url": "http://47.119.28.171",
    "port": 8081,
    "report_title": "自动化测试报告",
}

# 设置请求体的最大大小
DATA_UPLOAD_MAX_MEMORY_SIZE = 52428800  # 50M

# ================================================= #
# ************** 监控告警企微机器人配置  ************** #
# ================================================= #
QY_WEB_HOOK = ""

# ================================================= #
# ************** 发送邮件配置  ************** #
# ================================================= #
# 使用 SMTP 服务器发送邮件
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
# SMTP 服务器地址
EMAIL_HOST = "smtphz.qiye.163.com"
# SMTP 服务器端口
EMAIL_PORT = 465
# 发件人邮箱账号
EMAIL_HOST_USER = ""
DEFAULT_FROM_EMAIL = ""
# 发件人邮箱密码
EMAIL_HOST_PASSWORD = ""
# 是否使用 TLS
EMAIL_USE_TLS = False
# 是否使用 SSL
EMAIL_USE_SSL = True

# ================================================= #
# ************** 录制流量代理配置  ************** #
# ================================================= #
# PROXY Server
PROXY_ON = True  # 是否开启代理
PROXY_PORT = 7778

Jenkins构建部署

deploy10

deploy11

自有测试平台LunarLink部署生产环境实践

# 原理

# 准备工作

# 操作步骤

# Docker

# Jenkins

# Nginx

# 前端

# 后端

# Jenkins构建部署

ON THIS PAGE

原理