【2022鐵人賽】BuildCode YAML拆解task為step範本

前一篇「初探YAML的範本(Template)設計」,我們快速的將原本CI Pipeline中的BuildCode Job拆出來成為獨立的YAML檔案,然後以template的方式來使用它,因為是以jobs開頭,就是以job為主,所以可以把它稱作為Job範本(Job template)。

同樣的,如果再回去看一下「再一次認識Build Pipeline的YAML結構」這篇文章,可以發現jobs上面還有stages,jobs的下面則是steps,也就是說stage和step(task)都可以拆解出來成為範本,只是需要思考一下拆解到什麼程度比較適合。

讓我們再看一次BuildCode Job裡面的YAML內容


jobs:
  - job: BuildCode
    steps:
      - checkout: sources
        clean: true
      - task: Bash@3
        displayName: Check folder exist or create
        inputs:
          targetType: 'inline'
          script: |
            if [ -d "$(Build.ArtifactStagingDirectory)/pipelineFiles/" ]; then echo "$(Build.ArtifactStagingDirectory)/pipelineFiles/ exist"; else mkdir -p "$(Build.ArtifactStagingDirectory)/pipelineFiles/"; fi
      - task: PowerShell@2
        displayName: 'Get git commit sha'
        inputs:
          targetType: 'inline'
          workingDirectory: $(Build.SourcesDirectory)
          pwsh: true
          script: |
            $gitFullCommitSHA = git rev-parse HEAD
            $gitCommitSHA = $gitFullCommitSHA.Substring(0,8)

            echo "git commit sha: $gitCommitSHA"
            echo "git full commit sha: $gitFullCommitSHA"

            echo "$gitFullCommitSHA" > $(Build.ArtifactStagingDirectory)/pipelineFiles/gitCommitSHA.txt
      - task: PublishBuildArtifacts@1
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/pipelineFiles'
          ArtifactName: 'PipelineFiles'
          publishLocation: 'Container'
      - script: |
          export UID=$(id -u)
          export GID=$(id -g)
          docker run --user $UID:$GID --rm \
          -v $(Build.SourcesDirectory):/tmp/source \
          -v $(Build.BinariesDirectory):/tmp/publish \
          -e DOTNET_CLI_HOME=/tmp/.dotnet \
          mcr.microsoft.com/dotnet/sdk:6.0-alpine \
          dotnet publish /tmp/source/$(slnOrCsprojName) \
          -c release \
          -o /tmp/publish
        displayName: Dotnet Build
      - task: ArchiveFiles@2
        displayName: 壓縮成zip
        inputs:
          rootFolderOrFile: $(Build.BinariesDirectory)
          includeRootFolder: false
          archiveType: 'zip'
          archiveFile: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
          replaceExistingArchive: true
      - task: PublishBuildArtifacts@1
        displayName: 上傳到Pipeline Artifact
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
          ArtifactName: '$(pipelineArtifact)'
          publishLocation: 'Container'

在這裡面其實可以分成四種類型的steps:

  1. 取得git CommitSHA
  2. 上傳成品到Pipeline Artifacts
  3. Build Source Code
  4. 壓縮檔案

上面這四種都可以拆解出來成為獨立的step template,透過前面「認識Build Pipeline的參數(Parameters)與變數(Variables)」這篇介紹的Parameters來設計使用template時需要傳入的參數,這樣就可以讓範本重複使用在不同的情境,接下來就實際把上面的YAML再拆解成為更細的step template吧!

第一個我們先把上面第2個「上傳成品到Pipeline Artifacts」改成step template,因為第1個「取得git CommitSHA」最後也是要放到Pipeline Artifacts。

# steps/publish-pipeline-artifacts.yaml

parameters:
  - name: publishPath
    type: string
  - name: artifactName
    type: string
    default: OutputFiles

steps:
  - task: PublishBuildArtifacts@1
    displayName: 上傳成品到Pipeline Artifacts
    inputs:
      PathtoPublish: '${{ parameters.publishPath }}'
      ArtifactName: '${{ parameters.artifactName }}'
      publishLocation: 'Container'

「上傳成品到Pipeline Artifacts」只有一個task,而它需要的就是告訴它要發佈的檔案在哪裡,放上去的Artifact名稱要叫什麼,所以在參數的部份就設計了publishPath和artifactName,其中artifactName的部份我們設定了預設值,如果使用的時候沒有設定內容,那就直接使用OutputFiles為預設值。有些參數並不一定需要每一次都設定,有預設值可以在部份情況下減少輸入許多參數,就不會讓YAML內容看起來很長。

把task改成template之後,同樣的位置就要補上template關鍵字引用它,不過路徑的部份和前一篇的寫法有些不同。因為是在jobs/buildCode.yaml中要引用steps/publish-pipeline-artifacts.yaml,也就是上面拆解後的檔案存放的路徑,所以使用相對於jobs/buildCode.yaml路徑的方式引用(就是相對路徑啦)。

      - template: ../steps/publish-pipeline-artifacts.yaml
        parameters:
          publishPath: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
          artifactName: '$(pipelineArtifact)'

在template關鍵字下面使用parameters來指定要帶入的參數,每一個參數相對於parameters再多一層縮排。上面是將buildCode最後一個task改用template的結果,下面是原本使用task的內容,可以相互對照一下。

      - task: PublishBuildArtifacts@1
        displayName: 上傳到Pipeline Artifact
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
          ArtifactName: '$(pipelineArtifact)'
          publishLocation: 'Container'

在buildCode.yaml中有兩個上傳成品到Pipeline Artifacts的task,另一個我就不貼了,因為它準備一起變成另一個template了。

# steps/publish-commit-sha.yaml

parameters:
  - name: sourcePath
    type: string

steps:
  - task: Bash@3
    displayName: Check folder exist or create
    inputs:
      targetType: 'inline'
      script: |
        if [ -d "$(Build.ArtifactStagingDirectory)/pipelineFiles/" ]; then echo "$(Build.ArtifactStagingDirectory)/pipelineFiles/ exist"; else mkdir -p "$(Build.ArtifactStagingDirectory)/pipelineFiles/"; fi
  - task: PowerShell@2
    displayName: 'Get git commit sha'
    inputs:
      targetType: 'inline'
      workingDirectory: ${{ parameters.sourcePath }}
      pwsh: true
      script: |
        $gitFullCommitSHA = git rev-parse HEAD
        $gitCommitSHA = $gitFullCommitSHA.Substring(0,8)

        echo "git commit sha: $gitCommitSHA"
        echo "git full commit sha: $gitFullCommitSHA"

        echo "$gitFullCommitSHA" > $(Build.ArtifactStagingDirectory)/pipelineFiles/gitCommitSHA.txt
  - template: publish-pipeline-artifacts.yaml
    parameters:
      publishPath: '$(Build.ArtifactStagingDirectory)/pipelineFiles'
      artifactName: 'PipelineFiles'

上面的YAML是「取得git CommitSHA」改成step template的結果,可以發現裡面就不是單純只放了一個task,除了第一個task利用Bash確認我們要用的目錄是否存在與建立,第二個透過PowerShell來處理git commit sha的檔案,最後還用到了上面已經改為template的「上傳成品到Pipeline Artifacts」。

這裡有兩個地方要留意。

第一個就是參數的sourcePath在PowerShell task中設定給workingDirectory屬性,這個是對應git取得原始碼之後存放的位置。雖然之前是使用系統預先定義的變數$(Build.SourcesDirectory),但是在checkout的時候其實是可以設定path屬性,在多個不同的git repo取得原始碼的情況下,通常會使用path屬性讓它們放在不同的子目錄下,所以讓外部使用它的時候帶入正確的路徑是比較好的作法。

第二個就是在這裡面使用到前面一個publish-pipeline-artifacts範本,因為publish-commit-sha範本同樣是放在steps目錄底下,所以相對路徑就和上面在buildCode.yaml內不同。另外則是在publish-pipeline-artifacts範本的參數部份,因為兩個參數的資訊都在publish-commit-sha範本中都能決定,所以最終publish-commit-sha範本只有設計一個sourcePath參數,否則publish-pipeline-artifacts範本需要帶入的參數可能也要設計為publish-commit-sha範本的參數。

下面同樣將buildCode.yaml中改為template之後與之前分別貼上來對照參考。

      - template: ../steps/publish-commit-sha.yaml
        parameters:
          sourcePath: $(Build.SourcesDirectory)
      - task: Bash@3
        displayName: Check folder exist or create
        inputs:
          targetType: 'inline'
          script: |
            if [ -d "$(Build.ArtifactStagingDirectory)/pipelineFiles/" ]; then echo "$(Build.ArtifactStagingDirectory)/pipelineFiles/ exist"; else mkdir -p "$(Build.ArtifactStagingDirectory)/pipelineFiles/"; fi
      - task: PowerShell@2
        displayName: 'Get git commit sha'
        inputs:
          targetType: 'inline'
          workingDirectory: $(Build.SourcesDirectory)
          pwsh: true
          script: |
            $gitFullCommitSHA = git rev-parse HEAD
            $gitCommitSHA = $gitFullCommitSHA.Substring(0,8)

            echo "git commit sha: $gitCommitSHA"
            echo "git full commit sha: $gitFullCommitSHA"

            echo "$gitFullCommitSHA" > $(Build.ArtifactStagingDirectory)/pipelineFiles/gitCommitSHA.txt
      - task: PublishBuildArtifacts@1
        inputs:
          PathtoPublish: '$(Build.ArtifactStagingDirectory)/pipelineFiles'
          ArtifactName: 'PipelineFiles'
          publishLocation: 'Container'

接下來我打算將第3個「Build Source Code」和第4個「壓縮檔案」合併為一個template就好,因為壓縮檔案的部份以目前的設計來說,只有在編譯之後比較會用到,主要是為了減少成品上傳的大小和時間,如果編譯產生出來的檔案數量很多,沒有進行壓縮讓它只上傳少數的檔案,整體Pipeline執行的時間會在上傳到Pipeline Artifacts的時候花很多時間。

下面的YAML就是最後改為template的內容。

# steps/dotnet-build-in-linux-container.yaml

parameters:
  - name: buildImgRepository
    type: string
    default: mcr.microsoft.com/dotnet/sdk:6.0-alpine
  - name: srcPath
    type: string
  - name: slnOrCsprojName
    type: string
    default: Pipeline.sln
  - name: buildResultZipName
    type: string
    default: buildResult.zip
  - name: buildConfiguration
    type: string
    default: release
    values:
    - release
    - debug

steps:
  - script: |
      export UID=$(id -u)
      export GID=$(id -g)
      docker run --user $UID:$GID --rm \
      -v ${{ parameters.srcPath }}:/tmp/source \
      -v $(Build.BinariesDirectory):/tmp/publish \
      -e DOTNET_CLI_HOME=/tmp/.dotnet \
      ${{ parameters.buildImgRepository }} \
      dotnet publish /tmp/source/${{ parameters.slnOrCsprojName }} \
      -c ${{ parameters.buildConfiguration }} \
      -o /tmp/publish
    displayName: Dotnet Build
  - task: ArchiveFiles@2
    displayName: 壓縮成zip
    inputs:
      rootFolderOrFile: $(Build.BinariesDirectory)
      includeRootFolder: false
      archiveType: 'zip'
      archiveFile: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
      replaceExistingArchive: true

上面這個template中多了幾個不同的參數設計,其中一個是buildImgRepository,這個是因為我們是使用dotnet sdk的docker image,除了alpine的版本之外,也可以選用其它的版本,也可以改用不同dotnet版本的image,像是5.0、3.1。

template的設計基本上就是把內容中可以(或需要)改為參數設計的部份都設成參數讓外部使用時決定它的設定值。

同樣的,buildCode.yaml更改為使用template的前後內容如下。

      - template: ../steps/dotnet-build-in-linux-container.yaml
        parameters:
          srcPath: $(Build.SourcesDirectory)
          slnOrCsprojName: $(slnOrCsprojName)
      - script: |
          export UID=$(id -u)
          export GID=$(id -g)
          docker run --user $UID:$GID --rm \
          -v $(Build.SourcesDirectory):/tmp/source \
          -v $(Build.BinariesDirectory):/tmp/publish \
          -e DOTNET_CLI_HOME=/tmp/.dotnet \
          mcr.microsoft.com/dotnet/sdk:6.0-alpine \
          dotnet publish /tmp/source/$(slnOrCsprojName) \
          -c release \
          -o /tmp/publish
        displayName: Dotnet Build
      - task: ArchiveFiles@2
        displayName: 壓縮成zip
        inputs:
          rootFolderOrFile: $(Build.BinariesDirectory)
          includeRootFolder: false
          archiveType: 'zip'
          archiveFile: '$(Build.ArtifactStagingDirectory)/zipFiles/$(buildResultZipName)'
          replaceExistingArchive: true

最後替buildCode.yaml進行總整理,把各個template的參數整理一下變成buildCode這個Job template的parameters,其中在最後一個publish-pipeline-artifacts範本的部份稍微修改了一下publishPath參數,設定為目錄而不是檔案,也就是將整個目錄上傳,主要是因為在使用dotnet-build-in-linux-container範本的時候省略了設定zip檔名,所以最後buildCode.yaml的參數也只有兩個。

# jobs/buildCode.yaml

parameters:
  - name: sourcePath
    type: string
    default: $(Build.SourcesDirectory)
  - name: slnOrCsprojName
    type: string
    default: Pipeline.sln

jobs:
  - job: BuildCode
    steps:
      - checkout: sources
        clean: true
      - template: ../steps/publish-commit-sha.yaml
        parameters:
          sourcePath: ${{ parameters.sourcePath }}
      - template: ../steps/dotnet-build-in-linux-container.yaml
        parameters:
          srcPath: ${{ parameters.sourcePath }}
          slnOrCsprojName: $(slnOrCsprojName)
      - template: ../steps/publish-pipeline-artifacts.yaml
        parameters:
          publishPath: '$(Build.ArtifactStagingDirectory)/zipFiles'
          artifactName: '$(pipelineArtifact)'

因為buildCode.yaml增加了parameters的設定,所以在CI Pipeline的YAML中也要進行相對應的調整,就是在template底下設定需要的parameters:

# Pipelines git repository的ci/netapp-ci.yaml

jobs:
  - template: jobs/buildCode.yaml@templates
    parameters:
      sourcePath: $(Build.SourcesDirectory)
      slnOrCsprojName: $(slnOrCsprojName)

改完Pipeline的YAML後執行一次,BuildCode job全部綠燈沒問題。

發佈留言

%d 位部落客按了讚: