Key Concepts:

1. GitLab Runner: This is an application that runs your CI/CD jobs. It can be installed on different systems such as Linux, macOS, or Windows, or even within Docker containers.

2. Types of Runners:

   • Shared Runners: These are provided by GitLab and available to all projects on the instance. They are typically used for jobs that don't need a specific environment.

   • Specific Runners: These are set up for specific projects or groups. A specific runner is tied to a particular project or group, and you can configure it with custom settings or resources.

   • Group Runners: These are available to all projects within a specific GitLab group.

3. Runner Executors: The executor is the environment in which the job will run. Common executors include:

   • Shell: Executes jobs directly on the host machine's shell.

   • Docker: Runs jobs inside a Docker container.

   • Docker: Uses Docker Machine to create dynamic Docker containers for each job.

   • Kubernetes: Runs jobs in a Kubernetes cluster.

   • VirtualBox, Parallels, and others: Other executors exist that run jobs on virtual machines or other environments.

4. Job Execution: When a commit is made in a GitLab project, the runner listens for new jobs and picks them up to execute according to the instructions in the .gitlab-ci.yml file.

how to create own runner:

download gitlab runner package

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

install gitlab-runner

sudo apt install gitlab-runner

1. Go to Settings → CI/CD, in your project

2. Expand the Runners section

3. Select New project runner

Enter the details and click on create runner

select the operating system

then copy the token command and run it on runner Linux machine

gitlab-runner register
  --url https://gitlab.com
  --token glrt-t3_tpgWi1q2uWmBh6i2_q

Linux server terminal output

After that runner will connect to the GitLab server.

stages:
  - build
  - test
  - deploy

variables:
   name: "satish"

build_job:
     stage: build
     script:
       - echo "hello $name"
       - mkdir logs
       - echo "thiss are my result" >logs/app.log
     artifacts:
       paths:
           - logs/ 
       expire_in: 30 days


test_job:
     stage: test
     script:
       - echo "helloo $name"
     tags:
       - my-runner


deploy_job:
     stage: deploy
     script:
       - echo "heello $name"

using tags we can select the runner. here I am using my-runner tag to use newly created runner.

test_job:
     stage: test
     script:
       - echo "hello $name"
     tags:
       - my-runner

test job completed using my-runner

A GitLab pipeline is a powerful automation feature that helps automate the continuous integration and continuous delivery (CI/CD) process for your code. It defines a series of stages and jobs that execute in a predefined sequence, ensuring that your code is tested, built, and deployed efficiently.

Pipelines in GitLab are defined using a configuration file called .gitlab-ci.yml, which is stored in the root directory of the repository. The .gitlab-ci.yml file describes how the pipeline should run, including the jobs, stages, and scripts to be executed.

Key Concepts of GitLab Pipelines

1. Pipeline:

   • A pipeline is a collection of jobs organized into stages. A pipeline is triggered every time changes are made to the repository.

2. Stages:

   • Stages are the different steps in the pipeline. Each stage consists of one or more jobs that are executed concurrently, and once all jobs in a stage finish, the next stage begins.

   • Typical stages include:

     • Build: Compiling code or preparing assets.

     • Test: Running unit, integration, and other tests.

     • Deploy: Deploying the application to different environments (e.g., staging, production).

3. Jobs:

   • Jobs are the individual units of work in a pipeline. Each job runs a specific command or script (e.g., npm install, python test.py, docker build).

   • Jobs are executed on GitLab Runners, which are agents responsible for running the jobs.

4. GitLab Runners:

   • GitLab Runners are agents that run the jobs defined in the pipeline. They can be shared (available for all projects) or specific to a particular project or group.

5. Artifacts:

   • Artifacts are files generated by jobs (e.g., test results, compiled code, documentation) that are passed between stages or saved for later use.

6. Dependencies:

   • Jobs can depend on other jobs to run. This is managed using dependencies and ensures that a job only runs after certain jobs have completed successfully.

Step 1: Create a New GitLab Project

1. Log in to GitLab: Go to your GitLab instance and log in with your credentials.

2. Create a New Project:

   • On the GitLab dashboard, click the New project button.

   • Choose the type of project (e.g., Blank project).

   • Fill in the necessary details such as:

     • Project name: Choose a name for your project.

     • Visibility level: Select whether you want the project to be Private, Internal, or Public.

   • Click Create project.

Step 2: Set Up a GitLab CI Pipeline Configuration (.gitlab-ci.yml)

1. Navigate to your project: Once the project is created, you'll be taken to the project’s repository page.

2. Create the .gitlab-ci.yml file:

   • In the root directory of your project, create a file named .gitlab-ci.yml. This file contains the configuration for your CI pipeline.

Step 3: Commit the .gitlab-ci.yml File to Your Repository

1. Commit the .gitlab-ci.yml file:

   • Once the .gitlab-ci.yml file is created, commit it to your GitLab repository.

1. Job 3: deploy, which depends on test and also stores artifacts

    yamlCopy codestages:
      - build
      - test
      - deploy

    # Job to build the project
    build:
      stage: build
      script:
        - echo "Building the project..."
        - mkdir build_output
        - echo "Build successful" > build_output/build.log
      artifacts:
        paths:
          - build_output/
        expire_in: 1 hour

    # Job to test the project
    test:
      stage: test
      needs: 
        - job: build
          artifact: true
      script:
        - echo "Running tests..."
        - cat build_output/build.log
        - echo "Test passed"
      artifacts:
        paths:
          - test_reports/
        expire_in: 2 hours

    # Job to deploy the project
    deploy:
      stage: deploy
      needs:
        - job: test
          artifact: true
      script:
        - echo "Deploying the project..."
        - cat test_reports/test_report.txt
        - echo "Deployment completed"
      artifacts:
        paths:
          - deploy_logs/
        expire_in: 1 day

Step 4: Trigger the CI Pipeline

1. Push Changes to GitLab: Whenever you push the .gitlab-ci.yml file to your repository (or any other change), GitLab will automatically detect the presence of the .gitlab-ci.yml file and trigger the pipeline.

   • After you push the changes, go to the CI / CD section of the project, which can be found in the left sidebar under CI / CD.

2. Pipeline Started: You’ll see the pipeline is triggered and will move through the defined stages. If the pipeline doesn't trigger automatically, ensure that .gitlab-ci.yml is properly formatted and committed.

Step 5: Monitor CI Pipeline Jobs

1. Navigate to the Pipelines Page:

   • In your project, go to the CI / CD section from the left sidebar.

   • Click on Pipelines. This will show you a list of all pipelines for the project.

   • You’ll see the pipeline with a status indicating whether it's running, succeeded, or failed.

2. View Job Details:

   • Click on the pipeline entry to see detailed information about each job.

   • You will be able to see:

     • The status of each job (success, failed, canceled, etc.).

     • Logs for each job by clicking on the job name (e.g., build, test, deploy).

     • Artifacts that were created during each job.