Setting up a Sidra installation using the CLI tool¶
The following guide covers how to install a whole Sidra environment from scracth. To do so, a CLI tool is provided to ease the process. The purpose of this document is to guide through the needed actions that must be taken with the CLI to deploy a new and totally functional Sidra platform.
To be able to setup the complete Sidra environment we must own - An Azure subscription - An Azure DevOps account
In order to setup a working development environment, the following steps are needed
- Creation of AAD applications on Azure on each environment (development, test and production)with the needed permissions to allow the Sidra platform to access the required resources
- Setup of Azure DevOps
- Creation of Sidra solution source code repository
- Creation of CI/CD pipelines
- Setup of Azure DevOps variable groups for the CI/CD pipelines
In the following sections we are going to see how these steps are achieved and drastically simplified with the CLI tool
When the CLI is opened, it displays the available commands that can be used. To open it, just go to the directory where the executable is located, open a new CMD window, type
sidra and press enter.
All commands contains a description and can be asked for help in order to know what options that command can be passed. To request help for any command, just type
sidra <COMMAND> --help
For example, to know what parameters are needed to be passed to the
sidra install aadapplications command, type
sidra install aadapplications --help and the information will be displayed:
Setup with the CLI¶
Following are the details of each command to use with the CLI to setup a Sidra installation
Creating the Azure DevOps profile¶
Several commands through the process need to be passed the same information. To avoid this, there is an specific command that creates a profile with that common shared information between them, which is the
So, to start the process, create a new profile. You can display the information of the parameters that need to be passed using
sidra profile create --help.
Execute the command, filling the needed information:
sidra profile create --devOpsProjectUrl "<DEV_OPS_PROJECT_URL>" --devOpsPAT "<DEV_OPS_PAT>" --devOpsProject "<DEV_OPS_PROJECT>" --environment "<ENVIRONMENT>"
Installing the Active Directory applications¶
In order to work properly, Sidra needs to create several applications in the Azure Active Directory to access different resources, like access to the DataFactory APIs or Identity Server federation among others.
The applications are created per environment, so they are not shared between the different created environments.
The command will create the following:
- The AAD applications.
- An Azure Keyvault resource in which the sensitive information about the created applications will be stored.
- The Core Resource group, for the mentioned KeyVault.
Execute the command, filling the needed information. Remember that you can use the CLI help to know what each parameter is for.
sidra install aadapplications --tenant "<TENANT>" --subscription "<SUBSCRIPTION>" --location "<LOCATION>" --keyvaultName "<KEYVAULT_NAME>" --identityServerUrl "<IDENTITY_SERVER_URL>" --customerClientId "<CUSTOMER_CLIENT_ID>" --customerClientSecret "<CUSTOMER_CLIENT_SECRET>" --resourcegroup "<RESOURCEGROUP>"
The detail of the created applications can be found on the Managed Applications blade on Azure portal. The name of the applications will follow the pattern
Currently 3 applications are created
<Project>.<Env>.IdentityServerAdAccessfor Identity Server federation
<Project>.<Env>.Managerto manage operations on Azure resources, primarily needed for the Data Factory
<Project>.<Env>.VSTSto manage Azure DevOps and Azure resource deployments
Deploying the source code¶
The platform is deployed using different PowerShell scripts and ARM templates that contain the definition of all the Azure resources used by Sidra and the orchestration to deploy them. Apart from this, there are webjobs, an API and custom activities used in DataFactory.
All of them are dispatched as Nuget packages. These Nuget packages are then installed in the Sidra solution.
Finally, to perform the deployment of the mentioned above, a multi-stage CI/CD pipeline is deployed and configured in Azure DevOps.
sidra deploy source will be in charge of everything. It will:
- Create the Sidra solution and push all the code to the Azure DevOps default repository.
- Create the variable group for the environment which is being deployed in the Azure DevOps library, with the definition of all the parameters and default values for some of them.
- Create the multi-stage pipeline.
Execute the command, filling the needed information.
sidra deploy source
Alternatively, if the installation is going to use to the preview version of Sidra, the used command must be:
sidra deploy source --enablePrerelease
Custom naming convention¶
Before proceeding, if the organization uses a custom naming convention for Azure resources, it is needed to create two scripts and copy them to the Functions folder of the deployment project in the created solution. If there's no custom naming convention this step can be skipped.
These scripts are:
- GetCustomResourceName.ps1, which defines the naming that the deployed resources will have.
- GetCustomResourceTags.ps1, which defines the tags that the deployed resources will have.
They need to receive the same parameters as the GetDefaultResourceName.ps1 and GetDefaultResourceTags.ps1 scripts.
These files have to be placed on the following path on the code repository
To make these changes effective, these files have to be included in the generated binaries. To do so we can do it either with Visual Studio or editing the project file. After editing, commit and push the changes into the remote git repository.
Open the Sidra solution file on the root folder
PlainConcepts.Sidra.sln and follow these steps
- On the Solution Explorer window, unfold the
- Right click and select Properties
- On the Properties panel, on the property Copy to Output Directory select Copy always
- Save the project changes
Project file edit¶
Open deployment project file
.\src\PlainConcepts.Sidra.Deployment\PlainConcepts.Sidra.Deployment.csproj with the text editor of your choice and add the following xml tags and values to the file.
1 2 3 4 5 6 7 8
In the previous step, the command created several variable groups with the definition of the parameters that are used to deploy the platform.
This last command will fill the parameters to finally deploy the solution. Many of them contain default values that can be overwritten.
You can see all the configurable parameters that can be passed to the command with
sidra deploy settings --help.
The bare minimum are the indicated by the following command template. Fill the necessary information and execute it.
sidra deploy configure --deploymentOwnerName "<DEPLOYMENT_OWNER_NAME>" --deploymentOwnerEmail "<DEPLOYMENT_OWNER_EMAIL>" --infrastructureAlertEmail "<INFRASTRUCTURE_ALERT_EMAIL>" --companyName "<COMPANY_NAME>" --sidraAdminSubject "<SIDRA_ADMIN_SUBJECT>" --subscriptionId "<SUBSCRIPTION_ID>" --coreKeyVaultName "<CORE_KEY_VAULT_NAME>" --developmentTeam "<DEVELOPMENT_TEAM>" --resourceGroupName "<RESOURCE_GROUP_NAME>" --defaultDSUResourceGroupName "<DEFAULT_DSU_RESOURCE_GROUP_NAME>" --resourceNamePrefix "<RESOURCE_NAME_PREFIX>" --resourceNameMiddle "<RESOURCE_NAME_MIDDLE>" --installationSize "<INSTALLATION_SIZE>"
Now, everything is ready to deploy all the resources.
On Azure DevOps go to the Pipelines blade, and select the multi-stage pipeline "SIDRA
After running the pipeline Sidra will be ready to be used!
The process of setting up Sidra with the CLI is as simple as 4 commands. Each of these commands will be performed for each of the environments.
1 2 3 4 5 6
Update a Sidra installation¶
Apart from creating a Sidra project out of the box, the CLI allows to update the version of Sidra easily to the latest available, applying all needed updates to the Nuget packages, source code, DevOps settings and other possible tasks that are part of the newer versions.
update command is available starting in the 2020.R3 version, and will work from that point forward, so it is not possible to update previous versions using this command.
Depending on the current version installed, a series of tasks will be applied.
For example, let's say that the current installed version is 2020.R3, and after that the versions 2020.R4 and 2021.R1 were release.
If we upgrade, tasks will be applied for 2020.R4 and then for 2021.R1.
However, if the current version was 2020.R4, only tasks for 2021.R1 would be applied.
To perform the update, execute the command
sidra deploy update to update to the latest stable version.
Alternatively, if the installation is going to be updated to the preview version of Sidra, the used command must be:
sidra deploy update --enablePrerelease