Syntax highlighting of 35ff99b ~( devops/giltab/ci)

= Gitlab-CI =

<<TableOfContents()>>

== Refs ==

[[https://docs.gitlab.com/ce/ci/README.html|GitLab Continuous Integration (GitLab CI/CD)|class=" moin-https"]]

[[https://docs.gitlab.com/runner/install/docker.html#docker-image-installation-and-configuration|Configuration of your jobs with .gitlab-ci.yml|class=" moin-https"]]

[[https://docs.gitlab.com/ee/ci/yaml/|Keyword references|class=" moin-https"]] | [[https://docs.gitlab.com/ee/ci/yaml/includes.html#merge-method-for-include|include merge|class=" moin-https"]]

[[https://docs.gitlab.com/ee/ci/variables/predefined_variables.html|Predefined variables|class=" moin-https"]] | [[https://docs.gitlab.com/ee/ci/variables/|Gitlab Ci/CD Variables|class=" moin-https"]] | [[https://docs.gitlab.com/ee/ci/variables/#cicd-variable-types|File Type|class=" moin-https"]]

[[https://docs.gitlab.com/runner/|GitLab Runner|class=" moin-https"]]

[[https://about.gitlab.com/2016/07/29/the-basics-of-gitlab-ci/|GitLab CI: Run jobs sequentially, in parallel or build a custom pipeline|class=" moin-https"]]

[[https://gitlab.isoit.ru/ci/lint|Linter|class=" moin-https"]]

[[https://docs.gitlab.com/ce/ci/yaml/#only-and-except-simplified|Regexp in only|class=" moin-https"]]

[[https://docs.gitlab.com/ee/user/application_security/container_scanning/|Container Scanning|class=" moin-https"]]

[[https://docs.gitlab.com/ee/ci/examples/|CI/CD Examples|class=" moin-https"]]

[[https://docs.gitlab.com/ee/api/|API Docs|class=" moin-https"]] |  [[https://docs.gitlab.com/ee/api/pipelines.html|Piplines|class=" moin-https"]] | [[https://docs.gitlab.com/ee/api/lint.html|Lint|class=" moin-https"]]

[[https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html|GitLab CI/CD artifacts reports types|class=" moin-https"]]

== Recipes ==

=== Workflow ===

[[https://medium.com/@steven.jackson/configuring-a-gitlab-ci-pipeline-for-rails-postgresql-and-rspec-57c411400172|Configuring a GitLab CI pipeline for Rails, PostgreSQL, rspec, and rubocop|class=" moin-https"]]

[[https://medium.com/@Empanado/simple-continuous-deployment-with-docker-compose-docker-machine-and-gitlab-ci-9047765322e1|Simple GitLab Containuous deployment with docker compose, docker machine and Gitlab CI|class=" moin-https"]] | [[https://hub.docker.com/r/jonaskello/docker-and-compose/|Образ для статьи|class=" moin-https"]]

{{{
workflow:
  rules:
    - if: $CI_MERGE_REQUEST_TITLE =~ /^WIP:/
      when: never
    - if: $CI_MERGE_REQUEST_ID
    - if: $CI_COMMIT_BRANCH == "master"
    - if: $CI_PIPELINE_SOURCE == "schedule"
    - if: $CI_PIPELINE_SOURCE == "web"

}}}

=== Rollback ===

Откат деплоя, если он завершился с ошибкой

{{{
deploy_job:
  stage: deploy
  script:
    - kubectl apply -f frontend-deployment.yaml
    - kubectl rollout status deployment/frontend --timeout=60s

rollback_deploy_job:
  stage: rollback
  script:
    - kubectl rollout undo deployment/frontend
  when: on_failure

}}}

== docker-образ ==

Можно использовать CI для сборки docker-образа. Для этого необходимо чтобы ''runner'' функционировал в ''привилегированном'' режиме:

{{{
sudo cat /srv/gitlab-runner/config/config.toml
# ...
#[[runners]]
#  name = "runner-2"
#  url = "https://gitlab.isoit.ru"
#  token = "123123..."
#  executor = "docker"
#  [runners.docker]
#    tls_verify = false
#    image = "docker:18.04"
#    privileged = true   <------------------------
#    disable_cache = false
#    volumes = ["/cache"]
#    shm_size = 0
#  [runners.cache]

}}}

Если `privileged = false`, то необходимо руками изменить значение и перезапустить ''runner''-контейнер.

В ''gitlab-ci''конфигурации, приведенной ниже, используется образ ''Docker in docker (did)''. Контейнер, запущенный из данного образа, как раз собирает ''внутри себя'' образ на этапе ''build''. Если сразу не "отгрузить" собранный образ в ''registry'', то он будет удален вместе с контейнером ''did''. Другими словами, мы получим лишь факт того, что образ может быть собран по ''Dockerfile''.

== Troubleshooting ==

Если при запуске сценария в ''runner'' выдается

{{{
docker info  
# Cannot connect to the Docker daemon at tcp://localhost:2375. Is the docker daemon running?

}}}

то необходимо скорректировать файл конфигурации ''runner''-а

{{{
#/srv/gitlab-runner/config/config.toml
...
[[runners]]
  name = "runner-2"
  url = "https://gitlab.isoit.ru"
  token = "68ab9e5d61d96660616e7b385e7257"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "docker:18.04"
    privileged = true
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] # <----------
    shm_size = 0
  [runners.cache]

}}}

и перезапустить соответствующий контейнер ''runner''

[[https://stackoverflow.com/questions/42867039/gitlab-ci-runner-cant-connect-to-unix-var-run-docker-sock-in-kubernetes|stackoverflow|class=" moin-https"]]

== Образец конфигурации ==

{{{
stages:
  - test
  - build

test:
  stage: test
  image: "phusion/passenger-ruby25:0.9.30"
  services:
    - postgres:10.3

  variables:
    POSTGRES_DB: platform_test
    POSTGRES_USER: postgres
    POSTGRES_PASSWORD: dbpassword

  before_script:
    - apt update -qq && apt install -y -qq libpq-dev
    - ruby -v
    - which ruby
    - gem install bundler --no-ri --no-rdoc
    - RAILS_ENV=test bundle install --local
    - cp config/database.yml.gitlab-test config/database.yml
    - RAILS_ENV=test bundle exec rake db:create db:schema:load

  script:
    - RAILS_ENV=test bundle exec rspec

  only:
    - develop

build:
  stage: build
  image: "docker:stable"
  # When using dind, it's wise to use the overlayfs driver for
  # improved performance.
  variables:
    DOCKER_DRIVER: overlay2
  services:
    - docker:dind
  before_script:
    - docker info
  script:
    - docker build -t waltix-platform .
  only:
    - develop

}}}

== Запуск runner-а ==

https://docs.gitlab.com/runner/install/docker.html

https://docs.gitlab.com/runner/configuration/advanced-configuration.html

=== Ruby ===

{{{
sudo mkdir -p /srv/gitlab-runner/ruby25/config

docker run -d --name gitlab-runner-ruby25 \
    --restart always \
    -v /srv/gitlab-runner/ruby25/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:11.8

docker exec -it gitlab-runner-ruby25 gitlab-runner register

}}}

Файл конфигурации 
''/srv/gitlab-runner/ruby25/config/config.toml''

{{{
concurrent = 4
check_interval = 0

[[runners]]
  name = "ruby25-passenger"
  url = "https://gitlab.isoit.ru"
  token = "secret"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "registry.isoit.ru:5000/cicdd/img/pr25"
    privileged = false
    disable_cache = false
    volumes = ["/cache", "/bundle"]
    shm_size = 0
  [runners.cache]

}}}

=== Docker ===

{{{
sudo mkdir -p /srv/gitlab-runner/docker/config
docker run -d --name gitlab-runner-docker \
    --restart always \
    -v /srv/gitlab-runner/docker/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:11.8
docker exec -it gitlab-runner-docker gitlab-runner register

}}}

Файл конфигурации 
''/srv/gitlab-runner/docker/config/config.toml''

{{{
concurrent = 2
check_interval = 0

[[runners]]
  name = "docker"
  url = "https://gitlab.isoit.ru"
  token = "secret"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "registry.isoit.ru:5000/cicdd/img/docker:stable"
    privileged = false
    disable_cache = false
    volumes = ["/cache", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
  [runners.cache]

}}}

=== Docker Compose ===

{{{
sudo mkdir -p /srv/gitlab-runner/docker-compose/config
docker run -d --name gitlab-runner-docker-compose \
    --restart always \
    -v /srv/gitlab-runner/docker-compose/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:11.8
docker exec -it gitlab-runner-docker-compose gitlab-runner register

}}}

Файл конфигурации 
''/srv/gitlab-runner/docker-compose/config/config.toml''

{{{
concurrent = 1
check_interval = 0

[[runners]]
  name = "docker-compose"
  url = "https://gitlab.isoit.ru"
  token = "b16382f2cbbe52f4f05b9da6774916"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "registry.isoit.ru:5000/cicdd/img/docker-and-compose"
    privileged = false
    disable_cache = false
    volumes = ["/cache", "/minio:/minio", "/var/run/docker.sock:/var/run/docker.sock"]
    shm_size = 0
  [runners.cache]

}}}

=== Python ===

{{{
sudo mkdir -p /srv/gitlab-runner/python36/config
docker run -d --name gitlab-runner-python36 \
    --restart always \
    -v /srv/gitlab-runner/python36/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:11.8
docker exec -it gitlab-runner-python36 gitlab-runner register

}}}

Файл конфигурации ''/srv/gitlab-runner/python36/config/config.toml''

{{{
concurrent = 2
check_interval = 0

[[runners]]
  name = "python36"
  url = "https://gitlab.isoit.ru"
  token = "secret"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "registry.isoit.ru:5000/cicdd/img/py36"
    privileged = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  [runners.cache]

}}}

=== Node ===

{{{
sudo mkdir -p /srv/gitlab-runner/node9/config
docker run -d --name gitlab-runner-node9 \
    --restart always \
    -v /srv/gitlab-runner/node9/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:11.8
docker exec -it gitlab-runner-node9 gitlab-runner register

}}}

== Запуск runner-а локально ==

https://substrakt.com/how-to-debug-gitlab-ci-builds-locally/
https://docs.gitlab.com/runner/install/linux-manually.html

Предполагается, что Docker уже установлен

{{{
# скачиваем исполняеиый файл runner-а
sudo wget -O /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
# назначаем права на запуск
sudo chmod +x /usr/local/bin/gitlab-runner
# создаем специального пользователя
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash
# предположение:
# runner как сервис устанавливать необязательно, 
# т.к. мы будем пользовать его "вручную"
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner 
sudo gitlab-runner start
# --- Запуск этапа сборки (stage)
# выполнять из директории в которой лежит .gitlab-ci.yml
sudo gitlab-runner exec docker test

}}}

По всей видимости, при запуске команды `gitlab-runner exec` игнорируется файл с настройками `config.tomp`. Поэтому необходимо конфигурацию передавать через аргументы...

Кроме того необходимо учитывать, что каждая операция выполняется в собственной изолированной среде. Общие данные необходимо "складировать" в явно указанный том, в примере ниже это ''/cache''.

{{{
gitlab-runner exec docker build \
    --docker-volumes /cache:/cache \
    --docker-volumes /var/run/docker.sock:/var/run/docker.sock \
    --docker-privileged=true

}}}

На данный момент при локальном запуске runner-а нет доступа к секретным переменным. Как временное решение - использование секции ''variables''.

{{{
build:
  stage: build

  image: "registry.isoit.ru:5000/deployer/img/docker:stable"

  # When using dind, it's wise to use the overlayfs driver for
  # improved performance.
  variables:
    MCI: registry.isoit.ru:5000/cicdd/img/mc:latest
    MS: http://10.0.100.234:9000
    # MINIO_USER: secret variable
    # MINIO_PASSWORD: secret variable
    DOCKER_DRIVER: overlay2
    DOCKER_REGISTRY: registry.isoit.ru:5000
    # DOCKER_PASSWORD: secret variable
    BUILD_IMAGE: registry.isoit.ru:5000/zoid/waltix-platform-demo:develop

  services:
    - "registry.isoit.ru:5000/deployer/img/docker:dind" 

  before_script:
    - docker info
    - docker run --rm -v /cache/.mc:/root/.mc $MCI config host add minio $MS $MINIO_USER $MINIO_PASSWORD

  script:
    - docker run --rm -v /cache/.mc:/root/.mc -v /cache/tmp:/tmp $MCI cp minio/ds4-config/.vimrc /tmp
    - docker run --rm -v /cache/.mc:/root/.mc -v /cache/tmp:/tmp $MCI find minio/docker-certs --name "*.pem" --exec "mc cp {} /tmp" 
    - ls -la /cache/tmp
    - docker login -u zoid -p $DOCKER_PASSWORD $DOCKER_REGISTRY
    - docker build -t $BUILD_IMAGE  .
    - docker push $BUILD_IMAGE

  only:
    - develop

}}}

== Gitlab Coverage Badge ==

https://about.gitlab.com/2016/11/03/publish-code-coverage-report-with-gitlab-pages/

== Troubleshooting ==

=== unknown command "sh" for "some-command" ===

При запуске Helm скрипта в раннере получаем ошибку

{{{
Error: unknown command "sh" for "helm"

}}}

Проблема описана здесь - https://stackoverflow.com/questions/67289837/gitlab-issue-while-running-helm-command-as-error-unknown-command-sh-for-he
Необходимо удалить entrypoint для job

{{{
package:
  stage: build
  image:
    name: alpine/helm:3.7.1
    entrypoint: [""]
  script:
    - helm package .
  # ...

}}}

=== container is marked for removal ===

{{{
ERROR: Preparation failed: Error response from daemon: container is marked for removal and cannot be started

}}}

https://gitlab.com/gitlab-org/gitlab-runner/issues/2400