The Sidra platform is deployed using different PowerShell scripts and Azure Resource Manage (ARM) templates. The ARM templates contain the definition of all the Azure resources used by Sidra and the orchestration. With the Azure resources, there are Webjobs and an API. All of these PowerShell scripts and ARM templates are provided as Nuget packages. These Nuget packages are then installed in the Sidra solution.
Finally, to perform the deployment of all the above-mentioned resources in the previously created Git repository, a multi-stage CI/CD pipeline is created and configured in Azure DevOps. The command
sidra deploy source will take care of all these actions; see details below.
The usage options are source, configure and update options.
.\sidra.exe deploy source [parameters] .\sidra.exe deploy configure [parameters] .\sidra.exe deploy update [parameters] .\sidra.exe deploy [source|configure|update] --help
1. Source Option¶
This command is copying the code - PowerShell scripts and ARM templates - that will deploy the Sidra solution in Azure.
Taking the created configuration, recorded with the create profile, and the initial resources, created by install aadapplications, the
sidra deploy source command will:
- Create the Sidra solution and push all the code to the Azure DevOps provided repository.
- Create the variable group for the environment that is being deployed in the Azure DevOps library. The environment variable groups include the definition of all the parameters and default values for some of them.
- Create the Azure DevOps multi-stage pipeline.
Before running the command, check the PAT - the DevOps Personal Access Token. Make sure it is correct, still valid, and filled in the deployment profile.
See the profile command!
sidra deploy source, please ensure that there is no folder
PlainConcepts.Sidra within the Sidra.exe CLI folder.
The path where the project is published - or where the repo has been cloned if done from
/bin folder - can generate a path that is too long. This would prevent loading some projects of the generated solution (PlainConcepts.Sidra): the command could fail. To avoid this, try to choose a root folder whose path is as short as possible.
Also, if the machine from where the command runs is "fresh", make sure that Git is properly configured in that same machine:
git config --global user.email "John.Doe@AdventureWorks.com" ` git config --global user.name "John Doe"
false. It may be present (set to
true) or absent (defaulting to
false) in the command. It may be present (set to
true) or absent (defaulting to
false) in the command.
As per a current limitation, when trying to make a first deployment of Sidra, the steps to create the infrastructure for the web apps (Identity Server, Balea Server, FrontEnd) may throw an error message. This is just a temporary issue; with the next installation attempt, the resources will be found.
2. Configure option¶
This command option is filling in the values for the parameters needed before the installation's last step - execution of the Sidra deployment pipeline from Azure DevOps.
sidra deploy source command created several variable groups, with the definition of the parameters that are used to deploy the platform. The
sidra deploy configure command will fill-in the parameters to finally deploy the solution.
Many of the parameters below contain default values that can be overwritten later, so it is not required to populate all parameters.
You can see all the configurable parameters that can be passed to this command by typing
sidra deploy [source|configure|update] --help as seen above.
The main parameters are the indicated by the following command template. Fill in the necessary information and execute it:
We could have:
sds: arbitrary string, maximum 4 characters, chosen at the Sidra installation (resourceNamePrefix); here, we chose the acronym for PlainConcept Sidra Installation.
core: arbitrary string, maximum 4 characters, chosen at the Sidra installation (resourceNameMiddle); here, we chose it indicating this is for Sidra Core.
devis the environment name, shortening from development.
kvais a hardcoded particle, a fixed string, added automatically at Sidra's deployment, signifying Key Vault.
sds (here, acronym for PlainConcept Sidra Installation).
Lfor Large. Optional.
Determines the useraname to use when accessing the Llagar Azure Container Registry (ACR) that is hosting images. Required. Example:
true, a Batch Account will be deployed and a Linked Service attached to it will be created in the DSU´s Data Factory. Default value is
SMTP configuration (required)¶
The SMTP is used in the Identity Server to send emails to reset passwords or create new passwords, user registration, etc. There are two possible options to configure SMTP.
1. Use of SendGrid¶
By default, Sidra's deployment is configured to allow the use of SendGrid. If using SendGrid, the following paramaters must be considered in order to configure it properly.
The configuration of the following parameters will be automatically defined:
2. Custom SMTP server¶
In the case that the user has its own SMTP server, then, it will be necessary to set up the following parameter as
false, otherwise, a new SendGrid account will be created.
When the parameter
--createSendGridAccount is set up as
false as before, the user will need to configure manually the parameters shown previously:
SendGrid account configuration
After the creation of the SendGrid account during the deployment, manual configuration is needed to set it up:
- From Azure Portal, navigate to Saas page and select your SendGrid account (
- Click on Configure account now and fill the form to activate your account.
- After this is done, from SendGrid publisher's site, under settings, click on API keys to create a new apikey (
- With the value of the apikey, set a new setting in your API appservice with name
Mailing:Smtp:Passwordwith that value. Finally, go to Settings/Sender Authentication and set your sender identity.
More detailed information about Microsoft Azure and SendGrid configuration is shown in the official documentation of SendGrid.
.\sidra.exe deploy configure ` --deploymentOwnerName="John Smith" ` --deploymentOwnerEmail="John.Smith@Northwind.com" ` --infrastructureAlertEmail="Sidra-Alerts@Northwind.com" ` --companyName="Northwind Works" ` --sidraAdminSubject="John Smith" ` --subscriptionId="GUIDGUID-GUID-GUID-GUID-GUIDGUIDGUID" ` --coreKeyVaultName="sds-core-dev-kva" ` --developmentTeam="Sidra" ` --resourceGroupName="Sidra.Core.Dev" ` --defaultDSUResourceGroupName="Sidra.DSU.Dev" ` --resourceNamePrefix="sds" ` --resourceNameMiddle="core" ` --installationSize="M" `
3. Update option¶
This option complements the
configure option, taking much the same parameters: it updates the existing variable groups for an already deployed Sidra installation.
The option is used for easily upgrading the Sidra solution to a newer or latest version available. With just a simple command, the Sidra.exe CLI tool is upgrading the Nuget packages, updating source code, DevOps settings and other possible tasks that are part of the newer versions.
Depending on the current version installed, different sets of tasks will be applied. The option is able to update over multiple versions up to the latest. For example, let's consider that version 2020.R3 was initially installed; and, after that, versions 2020.R4 and 2021.R1 were released:
- When upgrading with the CLI tool, tasks would be applied to upgrade to 2020.R4 and then for upgrading to 2021.R1.
- If the current version was 2020.R4, only tasks for upgrading to 2021.R1 would be applied.
sidra deploy update to upgrade to the latest stable version. This will update the source code stored in the repository and set a new version for the Sidra platform. In order for this command to be effective, the DevOps pipeline needs to be executed. This will actually download the most recent packages for the new version.
Upgrading from 2022.R1 (1.11) or lower versions, to 2022.R2 (1.12), will require a .NET Core 3.1.1XX SDK version.
There are no additional considerations to be taken for executing this command, other than the pre-requirements described above. The DevOps variable group values will be updated as well with this command. In order to verify the version that is being installed, there is a file named
sidra-cli.txt that contains the current version.
sidra deploy update counts with the following mandatory parameters:
You can see all the configurable parameters that can be passed to this command by typing
sidra deploy update --help.
4. Privatewebapps option¶
This command is optional, thus it is not necessary to run it from a normal Sidra installation. The only different aspect regarding a normal installation is the automatic addition of a new variable in Sidra Common DevOps var group:
privateWebAppsVnet. Taking the value of this variable, the DevOps pipeline decides which deployment perform in runtime. If the var is empty, a normal deployment will be run, if not, it will try to deploy private endpoints.
The list of private webapps in Sidra Core, for which the command will create the private endpoints, is:
- IdentityServer web app
- Balea Server web app
- Core web app API
- Core web app frontend
This is the subnet where the private endpoints will be registered. This is used for the private web apps inbound calls inside the VNET.
.\Sidra.exe deploy privatewebapps ` --privateWebAppsSubnetInbound = "webapps" ` --privateWebAppsSubnetOutbound = "webappsPriv" ` --privateWebAppsVnet = "sds-core-vnet-vn" ` --privateWebAppsVnetResourceGroup = "Sidra.Core.Vnet" ` --websiteCustomDomain = "apitest.sidra.dev" ` --frontendCustomDomain = "frontendtest.sidra.dev" ` --identityServerCustomDomain = "istest.sidra.dev" ` --baleaServerCustomDomain = "bstest.sidra.dev" ` --certificateThumprint = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" ` --agentPool = "Sidra VNET"
Prerequisites for updating Sidra to 2022.R3 (1.12.1)¶
As a previous requirement to the deployment of the private endpoints, the following operations must be done in the client VNET by the customer:
Creation of two subnets in the VNET, one for incoming traffic and another one for outgoing traffic (
privateWebAppsSubnetOutboundrespectively, parameters described above).
Registration of private IPs in the DNS
privatelink.azurewebsites.netzone (it must be created in the case it does not exist).
Important to take into account that private IPs can be registered previously to the private endpoints, by the fact that the IPs will be assigned by order of the webapps deployment with the following order:
Update Sidra to 2022.R3 (1.13) and deploy private endpoints and custom domains¶
To update a normal existing Sidra installation to include private endpoints and custom domains, the next steps must be followed. Bear in mind that there are some important aspects to consider regarding the creation of custom domains. For that, the Annex I must be followed.
deploy privatewebappscommand. Run the command described above. After that, the new variables will be defined inside the Common variable group, setting the environment ready for the deployment.
deploy updatecommand. This is the normal step to update any installation. This will modify the pipeline template
azurepipelines.ymlwith the new pipeline definition, among other changes in the deployment project and other templates used in the deployment.
Verify correct execution of pipeline. If previous steps went well, the variable
privateWebAppsVnetshould be present in DevOps library, so the pipeline should get the value in runtime, executing the new stages for deploying the private endpoints and custom domains.
3.1. Infrastructure stage
This stage will be mainly in charge of creating all the resources inside Sidra Core and DSU resource groups. Although it creates the private endpoints and custom domains, it is not necessary for it to be run within a self-hosted agent, given that does not need access to the private network.
3.2. Private webapps stage
The agent pool specified in the CLI command will be set as the pool used in this stage execution. This part will deploy app services code and start all the webjobs and applications, besides deploying DataFactory pipelines as the last step.
This stage needs to be run in a self-hosted agent given that it performs requests to the privatized app services. As a reminder, the agent is running in the VMSS previously mentioned, which belongs to the same VNET as the app services, allowing the requests.
The below picture is an example of an actual agent used for deployments. As you can see, by creating the VMSS inside the VNET, the instances (VMs) are created together with a Network Interface Card (NIC) with a private IP. Although this IP is not in the same subnet as the webapps, it is allowed to connect to them.