SaaS runners on Linux are autoscaled ephemeral Google Cloud Platform virtual machines.
Autoscaling means reduced queue times to spin up CI/CD jobs, and isolated VMs for each job, thus maximizing security. These shared runners are available on GitLab.com.
GitLab offers Ultimate tier capabilities and included CI/CD minutes per group per month for our All your CI/CD jobs run on
The gitlab-shared-runners-manager-X.gitlab.com fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don’t run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times.
These runners share a distributed cache through use of a Google Cloud Storage (GCS) bucket. Cache contents not updated within the last 14 days are automatically removed through use of an .
With SaaS runners on Linux, you can run commands in a CI
job before the runner attempts to run git init and git fetch to
download a GitLab repository. The
pre_clone_script
can be used for:
Seeding the build directory with repository data
Sending a request to a server
Downloading assets from a CDN
Any other commands that must run before the git init
To use this feature, define a CI/CD variable called
CI_PRE_CLONE_SCRIPT that contains a bash script.
The CI_PRE_CLONE_SCRIPT variable does not work on Windows runners.
Pre-clone script example
This example was used in the gitlab-org/gitlab project until November 2021.
The project no longer uses this optimization because the pack-objects cache
lets Gitaly serve the full CI/CD fetch traffic. See Git fetch caching.
The CI_PRE_CLONE_SCRIPT was defined as a project CI/CD variable:
(echo"Downloading archived master..."
wget -O /tmp/gitlab.tar.gz https://storage.googleapis.com/gitlab-ci-git-repo-cache/project-278964/gitlab-master-shallow.tar.gz
if[!-f /tmp/gitlab.tar.gz ];then
echo"Repository cache not available, cloning a new directory..."exit
fi
rm-rf$CI_PROJECT_DIRecho"Extracting tarball into $CI_PROJECT_DIR..."mkdir-p$CI_PROJECT_DIRcd$CI_PROJECT_DIRtar xzf /tmp/gitlab.tar.gz
rm-f /tmp/gitlab.tar.gz
chmod a+w $CI_PROJECT_DIR)
The first step of the script downloads gitlab-master.tar.gz from Google Cloud Storage.
There was a cache-repo
that was responsible for keeping that archive up-to-date. Every two hours on a scheduled pipeline,
it did the following:
Create a fresh clone of the gitlab-org/gitlab repository on GitLab.com.
Save the data as a .tar.gz.
Upload it into the Google Cloud Storage bucket.
When a job ran with this configuration, the output looked similar to:
$ eval"$CI_PRE_CLONE_SCRIPT"
Downloading archived master...
Extracting tarball into /builds/gitlab-org/gitlab...
Fetching changes...
Reinitialized existing Git repository in /builds/gitlab-org/gitlab/.git/
The Reinitialized existing Git repository message shows that
the pre-clone step worked. The runner runs git init, which
overwrites the Git configuration with the appropriate settings to fetch
from the GitLab repository.
CI_REPO_CACHE_CREDENTIALS must contain the Google Cloud service account
JSON for uploading to the gitlab-ci-git-repo-cache bucket.
Note that this bucket should be located in the same continent as the
runner, or .
concurrent=Xcheck_interval=1metrics_server="X"sentry_dsn="X"[[runners]]name="docker-auto-scale"request_concurrency=Xurl="https://gitlab.com/"token="SHARED_RUNNER_TOKEN"pre_clone_script="eval \"$CI_PRE_CLONE_SCRIPT\""executor="docker+machine"environment=["DOCKER_DRIVER=overlay2","DOCKER_TLS_CERTDIR="]limit=X[runners.docker]image="ruby:2.5"privileged=truevolumes=["/certs/client","/dummy-sys-class-dmi-id:/sys/class/dmi/id:ro"# Make kaniko builds work on GCP.][runners.machine]IdleCount=50IdleTime=3600MaxBuilds=1# For security reasons we delete the VM after job has finished so it's not reused.MachineName="srm-%s"MachineDriver="google"MachineOptions=["google-project=PROJECT","google-disk-size=25","google-machine-type=n1-standard-1","google-username=core","google-tags=gitlab-com,srm","google-use-internal-ip","google-zone=us-east1-d","engine-opt=mtu=1460", # Set MTU for container interface, for more information check https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3214#note_82892928"google-machine-image=PROJECT/global/images/IMAGE","engine-opt=ipv6", # This will create IPv6 interfaces in the containers."engine-opt=fixed-cidr-v6=fc00::/7","google-operation-backoff-initial-interval=2" # Custom flag from forked docker-machine, for more information check https://github.com/docker/machine/pull/4600][[runners.machine.autoscaling]]Periods=["* * * * * sat,sun *"]Timezone="UTC"IdleCount=70IdleTime=3600[[runners.machine.autoscaling]]Periods=["* 30-59 3 * * * *","* 0-30 4 * * * *"]Timezone="UTC"IdleCount=700IdleTime=3600[runners.cache]Type="gcs"Shared=true[runners.cache.gcs]CredentialsFile="/path/to/file"BucketName="bucket-name"