Skip to content

Commit

Permalink
ci: use GitHub token instead of PAT and push sbom, provenance to regi…
Browse files Browse the repository at this point in the history
…stry

- The Docker workflow file name has been changed from `docker-reused-setup-steps/action.yml` to `docker-reused-steps/action.yml`.
- The docker workflow title and description has been updated to indicate it is reusable.
- Removed `token` input from docker workflow.
- Added a comment about requirement of additional space due to large image size.
- Updated docker image login mechanism to use GitHub token instead of personal access token.
- Comment about GitHub Actions access management added.
- Reference to the setup file in `docker_publish.yml` updated to reflect filename change.
- Permissions for GitHub token have been defined in `docker_publish.yml`.
- Option to cache to registry instead of gha, in order to avoid capacity limit, has been added.
- Incorporated a flag for software bill of materials (`sbom`) and software provenance.
- Removed comments on potentially unneeded tools during disk space free up process.

Signed-off-by: 陳鈞 <[email protected]>
  • Loading branch information
jim60105 committed May 12, 2024
1 parent 041c30f commit a37e667
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 36 deletions.
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
name: Setup docker
name: Reusable docker workflow

description: Configure the docker workflow.
description: Reusable docker workflow.

inputs:
token:
description: "A GitHub PAT"
required: true
tag:
description: "A tag to use for the image"
default: "no_model"
Expand All @@ -21,23 +18,19 @@ outputs:
runs:
using: composite
steps:
# We require additional space due to the large size of our image. (~10GB)
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: true

# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
dotnet: true
haskell: true
large-packages: true
docker-images: true
swap-storage: true

- name: Docker meta data:${{ inputs.tag }}
- name: Docker meta:${{ inputs.tag }}
id: meta
uses: docker/metadata-action@v5
with:
Expand All @@ -47,14 +40,16 @@ runs:
type=sha,prefix=${{ inputs.tag }}-
type=raw,value=latest,enable=${{ inputs.tag == 'no_model' }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# Create a Access Token with `read:packages` and `write:packages` scopes
# CR_PAT
# You may need to manage write and read access of GitHub Actions for repositories in the container settings.
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ inputs.token }}
password: ${{ github.token }}
50 changes: 28 additions & 22 deletions .github/workflows/docker_publish.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Check this guide for more information about publishing to ghcr.io with GitHub Actions:
# https://docs.github.com/en/packages/managing-github-packages-using-github-actions-workflows/publishing-and-installing-a-package-with-github-actions#upgrading-a-workflow-that-accesses-ghcrio

# Build the Docker image and push it to the registry
name: docker_publish

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches:
- "master"
Expand All @@ -12,13 +14,15 @@ on:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
# Sets the permissions granted to the GITHUB_TOKEN for the actions in this job.
permissions:
contents: read
packages: write

jobs:
# Build the ubi-no_model without cache export
docker-ubi-no_model:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -27,7 +31,7 @@ jobs:

- name: Setup docker
id: setup
uses: ./.github/workflows/docker-reused-setup-steps
uses: ./.github/workflows/docker-reused-steps
with:
token: ${{ secrets.CR_PAT }}
tag: ubi-no_model
Expand All @@ -45,6 +49,7 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max

Expand Down Expand Up @@ -76,16 +81,17 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64, linux/arm64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max
sbom: true
provenance: true

# Run the no_model build first ensure that the code at least builds
docker-no_model:
# The type of runner that the job will run on
runs-on: ubuntu-latest
outputs:
digest: ${{ steps.publish.outputs.digest }}
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -94,9 +100,7 @@ jobs:

- name: Setup docker
id: setup
uses: ./.github/workflows/docker-reused-setup-steps
with:
token: ${{ secrets.CR_PAT }}
uses: ./.github/workflows/docker-reused-steps

- name: Build:no_model
uses: docker/build-push-action@v5
Expand All @@ -111,6 +115,7 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max

Expand Down Expand Up @@ -143,12 +148,14 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64, linux/arm64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max
sbom: true
provenance: true

# Download whisper model cache
docker-cache:
# The type of runner that the job will run on
runs-on: ubuntu-latest
strategy:
fail-fast: true
Expand All @@ -162,7 +169,6 @@ jobs:
- large-v3
needs: docker-no_model # wait for docker-no_model to finish

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -171,9 +177,8 @@ jobs:

- name: Setup docker
id: setup
uses: ./.github/workflows/docker-reused-setup-steps
uses: ./.github/workflows/docker-reused-steps
with:
token: ${{ secrets.CR_PAT }}
tag: cache-${{ matrix.model }}

- name: Build cache:${{ matrix.model }}
Expand All @@ -192,8 +197,11 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64, linux/arm64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max
sbom: true
provenance: true

# Run the rest of the builds in parallel
docker:
Expand Down Expand Up @@ -256,9 +264,8 @@ jobs:

- name: Setup docker
id: setup
uses: ./.github/workflows/docker-reused-setup-steps
uses: ./.github/workflows/docker-reused-steps
with:
token: ${{ secrets.CR_PAT }}
tag: ${{ matrix.model }}-${{ matrix.lang }}

- name: Get short SHA
Expand All @@ -284,23 +291,22 @@ jobs:
VERSION=${{ github.ref_name }}
RELEASE=${{ github.run_number }}
platforms: linux/amd64, linux/arm64
# Cache to regietry instead of gha to avoid the capacity limit.
cache-from: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache
cache-to: type=registry,ref=ghcr.io/${{ github.repository_owner }}/whisperx:cache,mode=max
sbom: true
provenance: true

test-large-v3-zh:
name: Test large-v3-zh docker image
runs-on: ubuntu-latest
needs: docker
steps:
# We require additional space due to the large size of our image. (~10GB)
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: true

# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
dotnet: true
haskell: true
Expand Down

0 comments on commit a37e667

Please sign in to comment.