Setting up a Sidra installation using the Sidra CLI tool¶
The following guide covers how to install a whole Sidra environment from scratch using the Sidra CLI tool.
Sidra CLI tool has been designed to ease the process of installing and updating Sidra Core.
This document details the different steps that must be taken with the CLI tool to deploy a new and totally functional Sidra Core platform environment.
To be able to setup the complete Sidra Core environment the following Azure resources must be owned:
- An Azure subscription
- An Azure DevOps account
Additionally, Sidra CLI includes some commands in order to check for pre-requisites for launching and executing the different installation commmands. These are detailed in the CLI overview section.
General Sidra Core installation process overview¶
The process of seting a working development environment for Sidra Core traditionally involves the following steps:
- Creation of AAD applications on Azure on each environment (e.g. development, test and production), with the needed permissions to allow the Sidra platform to access the required resources.
- Setup of Azure DevOps:
- Creation of the Sidra solution source code repository.
- Creation of the CI/CD pipelines for the code build and release steps.
- Setup of Azure DevOps variable groups for the CI/CD pipelines. Variable groups are a collection of variables which reside in the releases.
In the following sections we are going to see how these steps are achieved and drastically simplified with the CLI tool.
The CLI tool is a solution that needs to be compiled in the local environment from where it will be executed.
To open it, just go to the directory where the executable is located, open a new CMD window, type
sidra and press
When the CLI is opened, it displays the available commands that can be used:
All commands contain a description. Commmands can be executed with
sidra <COMMAND> --help to ask for help on the options available and needed parameters.
To request help for any command, just type
sidra <COMMAND> --help
For example, to know what parameters are needed to be passed to the
sidra install aadapplications command, type
sidra install aadapplications --help and the information will be displayed:
Sidra CLI tool requires to have some pre-requisites setup to run some commands.
There are some specific command options in the sidra CLI tool that allow to setup and validate prerequisites needed for executing commands.
On the sidra CLI tool, in the appropriate path where the executable is, type the following command to verify the pre-requisites for executing Sidra CLI:
sidra prerequisites verify
The following command will launch the pre-requisites installation:
sidra prerequisites setup
Seting up Sidra Core with the Sidra CLI installation tool¶
Following are the different steps involved in setting up a Sidra Core installation with the tool:
Step 1: Creating the Azure DevOps profile¶
The process of installing Sidra through the CLI tool requires to pass over the same information at different steps.
To avoid duplicating typing efforts, there is a specific command that creates a profile with the common shared information for a series of steps. This is the
The information on the parameters that need to be passed to this command is provided by typing
sidra profile create --help.
It is recommended create a new profile before starting an installation process. Execute the command, filling in the needed information:
sidra profile create --devOpsProjectUrl "<DEV_OPS_PROJECT_URL>" --devOpsPAT "<DEV_OPS_PAT>" --devOpsProject "<DEV_OPS_PROJECT>" --environment "<ENVIRONMENT>"
Step 2: Installing the Active Directory applications¶
In order to work properly, Sidra needs to create several applications in the Azure Active Directory to access different resources, like access to the Data Factory APIs or Identity Server federation, among others.
The applications are created per environment, so they are not shared between the different created environments.
The command to create the AAD applications will create the following:
- The AAD applications.
- An Azure KeyVault resource, which will hold the sensitive information about the created applications.
- The Core Resource group, for the mentioned KeyVault.
Execute the command, filling the needed information. Remember that you can use the CLI help to know what each parameter is for.
sidra install aadapplications --tenant "<TENANT>" --subscription "<SUBSCRIPTION>" --location "<LOCATION>" --keyvaultName "<KEYVAULT_NAME>" --identityServerUrl "<IDENTITY_SERVER_URL>" --customerClientId "<CUSTOMER_CLIENT_ID>" --customerClientSecret "<CUSTOMER_CLIENT_SECRET>" --resourcegroup "<RESOURCEGROUP>"
The detail of the created applications can be found on the Managed Applications blade on Azure portal. The name of the applications will follow the pattern
Currently 3 applications are created by this command:
<Project>.<Env>.IdentityServerAdAccessfor Identity Server federation.
<Project>.<Env>.Managerto manage operations on Azure resources, primarily needed for the Data Factory services.
<Project>.<Env>.VSTSto manage Azure DevOps and Azure resource deployments.
Step 3: Deploying the source code¶
The platform is deployed using different PowerShell scripts and ARM templates that contain the definition of all the Azure resources used by Sidra and the orchestration to deploy them. Apart from this, there are Webjobs, an API and custom activities used in DataFactory.
All of them are provided as Nuget packages. These Nuget packages are then installed in the Sidra solution.
Finally, to perform the deployment of all the above-mentioned resources, a multi-stage CI/CD pipeline is deployed and configured in Azure DevOps.
sidra deploy source will take care of all these actions. It will:
- Create the Sidra solution and push all the code to the Azure DevOps default repository.
- Create the variable group for the environment which is being deployed in the Azure DevOps library. The environment variable groups include the definition of all the parameters and default values for some of them.
- Create the Azure DevOps multi-stage pipeline.
Execute the command to deploy the source code by filling in the needed information:
sidra deploy source
Alternatively, if the installation is going to use to the preview version of Sidra, the used command must contain the
--enablePrerelease flag, as follows:
sidra deploy source --enablePrerelease
Step 4: Creating a custom naming convention (optional)¶
Before proceeding, if the organization uses a custom naming convention for Azure resources, two scripts need to be created to specifiy the naming conventions to use:
- GetCustomResourceName.ps1, which defines the naming that the deployed resources will have.
- GetCustomResourceTags.ps1, which defines the tags that the deployed resources will have.
They need to receive the same parameters as the GetDefaultResourceName.ps1 and GetDefaultResourceTags.ps1 scripts.
These files have to be placed on the following path on the code repository:
To make these changes effective, these files must be included in the generated binaries. This can be done by two alternative methods: with Visual Studio or by editing the project file. Both methods are detailed below.
After editing, commit and push the changes into the remote git repository.
OPTION A: Include scripts from Visual Studio properties panel¶
Open the Sidra solution file on the root folder
PlainConcepts.Sidra.sln and follow the following steps:
- On the Solution Explorer window, unfold the
- Right click and select Properties.
- On the Properties panel, on the property Copy to Output Directory select Copy always.
- Save the project changes.
OPTION B: Edit the Project file Project¶
Open deployment project file
.\src\PlainConcepts.Sidra.Deployment\PlainConcepts.Sidra.Deployment.csproj with the text editor of your choice and add the following xml tags and values to the file:
1 2 3 4 5 6 7 8
Step 5: Creating the final settings for the the deployment¶
In Step 3, the command created several variable groups with the definition of the parameters that are used to deploy the platform.
The command in this step will fill the parameters to finally deploy the solution. Many of them contain default values that can be overwritten later.
You can see all the configurable parameters that can be passed to the command by typing
sidra deploy settings --help.
The minimum set of mandatory parameters are the indicated by the following command template. Fill in the necessary information and execute it:
sidra deploy configure --deploymentOwnerName "<DEPLOYMENT_OWNER_NAME>" --deploymentOwnerEmail "<DEPLOYMENT_OWNER_EMAIL>" --infrastructureAlertEmail "<INFRASTRUCTURE_ALERT_EMAIL>" --companyName "<COMPANY_NAME>" --sidraAdminSubject "<SIDRA_ADMIN_SUBJECT>" --subscriptionId "<SUBSCRIPTION_ID>" --coreKeyVaultName "<CORE_KEY_VAULT_NAME>" --developmentTeam "<DEVELOPMENT_TEAM>" --resourceGroupName "<RESOURCE_GROUP_NAME>" --defaultDSUResourceGroupName "<DEFAULT_DSU_RESOURCE_GROUP_NAME>" --resourceNamePrefix "<RESOURCE_NAME_PREFIX>" --resourceNameMiddle "<RESOURCE_NAME_MIDDLE>" --installationSize "<INSTALLATION_SIZE>"
Step 6: Deploying Sidra Core!¶
At this step, all the parameters should be ready to deploy all the resources.
On Azure DevOps go to the Pipelines blade, and select the multi-stage pipeline "SIDRA
This pipeline will create the build artifacts and deploy the infrastructure and binaries on Azure.
After running the pipeline Sidra will be ready to be used!
The process of setting up Sidra with the CLI tool is as simple as typing 4 commands. Each of these commands will need to be performed for each of the environments.
1 2 3 4 5 6
Update a Sidra installation with the Sidra CLI tool¶
Apart from creating a Sidra project for the first time, the CLI allows to update the version of Sidra easily to the latest version availabl. In order to execute an update, the Sidra CLI tool is prepared to apply, just with one command, all the needed updates. This extends to: upgrading the Nuget packages, updating source code, DevOps settings and other possible tasks that are part of the newer versions.
Important notice: The
update command is available starting in the 2020.R3 version, and will work from that point forward, so it is not possible to update previous versions using this command.
Depending on the current version installed, different sets of of tasks will be applied.
Let's consider some example scenarios:
If the installed version is 2020.R3, and after that the versions 2020.R4 and 2021.R1 were released:
Then, when upgrading with the CLI tool, tasks would be applied for 2020.R4 and then for 2021.R1.
However, if the current version was 2020.R4, only tasks for 2021.R1 would be applied.
To perform the update, execute the command
sidra deploy update to update to the latest stable version.
Alternatively, if the installation is going to be updated to the preview version of Sidra, use the following command:
sidra deploy update --enablePrerelease