Link to home
Start Free TrialLog in
Avatar of Jasmin shahrzad
Jasmin shahrzad

asked on

gitlab yaml split stages

hi expert,
in gitlab i try to run som build stages, such as this build should running in all env. except for production i usedoption only (as it described in following doc: https://lab.las3.de/gitlab/help/ci/yaml/README.md)
it's not working 

variables:
  GIT_SUBMODULE_STRATEGY: "recursive"
  DOTNET_CLI_TELEMETRY_OPTOUT: "1"
  NVM_DIR: "/home/gitlab-runner/.nvm"

stages:
  - build
  - unittest
  - coverage
  - deploy

# Activate nvm if .nvmrc exists in project

before_script:
  - if [ -f .nvmrc ] ; then . $NVM_DIR/nvm.sh --no-use ; nvm install --no-progress ; else true ; fi

# Build Stage

dotnet-build:
  stage: build
  script: dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
  only:
    - ci
    - test
    - qa
#    - production
  tags:
    - dotnet
    - test

# Unittest Stage

dotnet-test:
  stage: unittest
  script: dotnet-test-all
  tags:
    - dotnet
    - test
.....

My issue is i have to scripts 

1-    script: dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE

AND

2-   script: dotnet build /clp:ForceConsoleColor 

 

I don't know how to say script to are for all env and script 2 are for all except production

Then i create 2 yaml file and change between file 1 and 2.

but file 2 (bellow)is orginal and ok for all env  but file 1 it's not working correct.

please advise.



ASKER CERTIFIED SOLUTION
Avatar of simon3270
simon3270
Flag of United Kingdom of Great Britain and Northern Ireland image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Avatar of Jasmin shahrzad
Jasmin shahrzad

ASKER

ok. maybe it's a answer.
But i am sure you can guide me to better solution.
Issue is script i have 2 exec in scripts
  1 - script: dotnet build /clp:ForceConsoleColor
  2 - script: dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
1- should work for all env and 2 should work for all except production
i don't know i can do it if condition here , therefore i create 2 file and developer should select file 1 or 2. 
How aboit setting a fact to the parameters for the script?
- set_fact: sc_parms=""

- set_fact: sc_parms='--configuration Release_SIMULATE'
  when: env == 'production'

Open in new window

Then call your script as
 script: "dotnet build /clp:ForceConsoleColor {{ sc_parms}}" 

Open in new window

You will have to use the appropriate variable for the "when" clause (I've used "env"), and I've also assumed that it will equal 'production'.
ok do you mean i need only one file? I don't know how to use in my yml file.
Should i use it after stage: build or where?  And it should not be 'production' sorry i am very new 

variables:
  GIT_SUBMODULE_STRATEGY: "recursive"
  DOTNET_CLI_TELEMETRY_OPTOUT: "1"
  NVM_DIR: "/home/gitlab-runner/.nvm"

stages:
  - build
  - unittest
  - coverage
  - deploy

# Activate nvm if .nvmrc exists in project

before_script:
  - if [ -f .nvmrc ] ; then . $NVM_DIR/nvm.sh --no-use ; nvm install --no-progress ; else true ; fi

# Build Stage

dotnet-build:
stage: build      #    <---- afterHere
  script: dotnet build /clp:ForceConsoleColor
 except
    - ci
    - test
    - qa
    - production
  tags:
    - dotnet
    - test

# Unittest Stage

dotnet-test:
  stage: unittest
  script: dotnet-test-all
  tags:
    - dotnet
    - test
..... 
Hi Jasmin,

I'm new to this too!

It's not standard Ansible, as far as I can tell - my fix was how you would do it in Ansible.

I'm still not entirely clear what you want. Do you want to run the command with the "Release_SIMULATE" parameter for all environments, but the one without "Release_SIMULATE" for all except production?

If that's true, and after looking at the document you linked to in your original question, I think you could have this where you just have a single "build" stage:
dotnet-build-simulate:
  stage: build
  script: dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
  only:
    - ci
    - test
    - qa
    - production
  tags:
    - dotnet
    - test

dotnet-build:
  stage: build
  script: dotnet build /clp:ForceConsoleColor
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

Open in new window

 "Do you want to run the command with the "Release_SIMULATE" parameter for all environments, but the one without "Release_SIMULATE" for all except production?"
 no revers i want  Release_SIMULATE  for all except prod and
without Release_SIMULATE for all env.
To clarify, do you want to run both jobs in ci, test and qa, but only the one without "Release_SIMULATE" in production. Does it matter which orde they run in?

I've given a structure which I think will work,with two separate job blocks. Feel free to modify it to do what you want.

In particular, if the two scripts are the wrong way round (i.e. you want to run the "SIMULATE" one second), move the entire "dotnet-build-simulate"  job block to after the "dotnet-build" one. I'm assuming here that gitlab runs both jobs, in the order specified, one after the other.

You can modify the 'only:" section in each job to run it in the correct environments (so take "- production" out of the simulate job, and add it to the non-simulate one).
Yes correct, want to run both jobs in ci, test and qa, but only the one without "Release_SIMULATE" in All 
env encloud  production. 
I don't want to run both in parallel or serie.
If there is a "Release_SIMULATE " then running in env test, ci, qa and without "Release_SIMULATE "
running in all env.
In general, if 2 "scripts" are called by the same job, then they either run in parallel or in series - you have to pick one! In fact, my suggested option of having 2 separate "build" jobs, gitlab will run them in parallel.

Stepping back, I think the overall aim of your gitlab YAML file is that if you are in ci, test or qa, you want to run 2 "scripts" - "dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE" and "dotnet build /clp:ForceConsoleColor". If you are in production, you only want to run "dotnet build /clp:ForceConsoleColor". From the parameter names, I am guessing that the non-SIMULATE call actually builds the target program, while the SIMULATE one checks that the built program will work in production.

In that case, you should have 2 separate build tasks. One runs in ci, test and qa, and runs both tasks, while the other runs in production and only runs the "dotnet build /clp:ForceConsoleColor" "script"
dotnet-build-nonprod:
  stage: build
  script: 
    - dotnet build /clp:ForceConsoleColor
    - dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

dotnet-build-prod:
  stage: build
  script: dotnet build /clp:ForceConsoleColor
  only:
    - production
  tags:
    - dotnet
    - test

Open in new window

As before, you can change the order things run in (e.g. swap round the SIMULATE and non-SIMULATE lines in the first task).

Note that gitlab will only run one of the tasks - in ci, test or qa (or "nonprod" as I have called it because it is anywhere that is not production), it will only run the first one. In production, it will only run the second one. So in nonprod it runs both "scripts", one after the other, while in production it only runs one "script".
No what i have is the following. you can see it's build independt of env. Therefor is except all env. they don't accept it's duild 2 (parallel or seie)

--------------
---
variables:
  GIT_SUBMODULE_STRATEGY: "recursive"
  DOTNET_CLI_TELEMETRY_OPTOUT: "1"
  NVM_DIR: "/home/gitlab-runner/.nvm"

stages:
  - build
  - unittest
  - coverage
  - deploy

# Activate nvm if .nvmrc exists in project

before_script:
  - if [ -f .nvmrc ] ; then . $NVM_DIR/nvm.sh --no-use ; nvm install --no-progress ; else true ; fi

# Build Stage

dotnet-build:
  stage: build
  script: dotnet build /clp:ForceConsoleColor
  except:
    - ci
    - test
    - qa
    - production
  tags:
    - dotnet
    - test
....
------------------
Then we will to run script: dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
on ci, test, qa.
i just think about your set_fact:  it should be solution (i think) but i don't know how to use it?
Does your YAML file work? It looks as though it will run the "dotnet build /clp:ForceConsoleColor" command but only if the branch isn't one of ci, test, qa and production. Are there any other branches?

The "set_fact" solution was for Ansible - this isn't Ansible!

Let's forget the YAML for a moment. What are you *actually* trying to do? I think that you want to run some programs if there is a change in the code (i.e. new code is checked into the ci, test, qa or production branches). For all of those 4 branches, you want to run the "dotnet build /clp:ForceConsoleColor" command. After that has completed successfully, you then want to run "dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE" but only if code is checked into the ci, test or qa branches.

Is that a fair summary? If so, my last code block will do that.
Yes it's work. what they do is build without any env and i think they select env when then deploy or something like that. I don't know how it's work. but they say it work and they want to other script with simulator running on ci, test and qa.
yes it's correct
 you say your last block do you mean:

dotnet-build-nonprod:
  stage: build
  script:
    - dotnet build /clp:ForceConsoleColor
    - dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

dotnet-build-prod:
  stage: build
  script: dotnet build /clp:ForceConsoleColor
  only:
    - production
  tags:
    - dotnet
    - test
The bit about the environment not being set is news!

So, if the environment is not set to ci, test, qa or production, you want to run the "dotnet build /clp:ForceConsoleColor" command (that is, the YAML file that you say is currently working).

Now when the environment is set to ci, test or qa, you want "dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE" to run. The question is, do you also want "dotnet build /clp:ForceConsoleColor" to run when ci, test or qa is set? And do you also want "dotnet build /clp:ForceConsoleColor" to run when production is set? And on ci, test and qa, do you want "dotnet build /clp:ForceConsoleColor" to run before or after "dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE"?

If that is all true (and with a possible change to the order of commands in the dotnet-build-nonprod task), yes, that code in your last comment is the section I think you should add, after the existing dotnet-build section.
it was correct i did it. but it not running for --configuration Release_SIMULATE . it's look like developer has an issue with this. because when i have only dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE ,
dose not create  Release_simulate. and read /clp:ForceConsoleColor and not the rest.
That's odd! What does your current script (in particular, any "stage: build" sections) look like now?
yes odd! right know is look like following. for test i remove donet-build-prod and in nonprod i have one script  --dotnet.. --configure...
it's buildwithout any error and don't create a  RiskAssessmentService.Simulator.dll 
it's create
"Executing "step_script" stage of the job script00:36
44$ if [ -f .nvmrc ] ; then . $NVM_DIR/nvm.sh --no-use ; nvm install --no-progress ; else true ; fi
45$ dotnet publish -c Release -o published-files "${APP_NAME}" /clp:ForceConsoleColor
46Microsoft (R) Build Engine version 17.1.0+ae57d105c for .NET"

ignor --configu....

dotnet-build-nonprod:
  stage: build
  script:
    - dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE
    - dotnet build /clp:ForceConsoleColor 
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

dotnet-build-prod:
  stage: build
  script: dotnet build /clp:ForceConsoleColor
  only:
    - production
  tags:
    - dotnet
    - test 
If complicated things don't work, making them simpler might help. Please test that the individual commands we are trying to run actually work. So, have this as your "stage: build" section
dotnet-build-nonprod:
  stage: build
  script:
    - "dotnet build /clp:ForceConsoleColor --configuration Release_SIMULATE"
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

Open in new window

When called with one of the relevant branches (ci, test or qa), it should run that command to do whatever that command does. If that works, try again with the other command:
dotnet-build-nonprod:
  stage: build
  script:
    - "dotnet build /clp:ForceConsoleColor"
  only:
    - ci
    - test
    - qa
  tags:
    - dotnet
    - test

Open in new window

Again, when code is pushed to the ci, test or qa branches, that should do whatever that command is supposed to do (remember, I have no idea what these commands actually do!). Only when both work should you try the one with 2 script lines.

By the way, was the complete output in your previous comment? The text "ignor --configu...." seems to be separate from the rest.

Also, I've put double-quotes round the commands - I don't think they are necessary, but they make it more obvious to the reader that the entire line is a command.