Getting Started with Supervisor on Azure¶

Information

Ensure you have a Sidra license before starting the process. For assistance or to obtain a license, please contact us at [email protected].

This document provides information for deploying the Supervisor which will include the Sidra deployment on Azure.

Prerequisites¶

The requirements for a proper deployment process are:

Access to an existing Azure Portal account is required before starting the deployment.
Download Supervisor's deployment files. PowerShell scripts (must all be contained in the same folder) for executing the pre-deployment script:
Supervisor's ARM Template json.
An Azure Portal user with these permissions:
1. Azure RBAC Built-In Roles scoped to the Subscription:
  - Contributor:
    - Allows to create and manage all types of Azure resources (keyVaults, container Apps, Storage accounts...).
    - Assigning Contributor role to the managed identity of Supervisor is mandatory to allow deploying Sidra Service from Supervisor.
  - Role Based Access Control Administrator (Preview) or User Access Administrator:
    - This role is needed to execute a PowerShell script from ARM template. This script resets the password of Active Directory App Registration created during the Pre-deploy script execution.
  Info
  
  Role Based Access Control Administrator (Preview) has fewer allowed actions and is recommended. However, in case of the preview doesn't allow this Role to be eligible to the customer, there's also the option of using the User Access Administrator role.
  
  These user access to subscription levels can be checked on Access control section on Azure portal.
2. Microsoft Entra ID (MEID) Roles (previously Azure Active Directory Roles):
  - Application Administrator. Allows to create and manage applications in Azure AD, including registering new applications and managing existing applications. It is used to create <<CompanyName>>.Supervisor.<<Environment>>.Manager and <<CompanyName>>.Supervisor.<<Environment>>.IdentityServerAdAccess.
  - Privileged Role Administrator. Allows to assign roles to other users, groups, or applications in the subscription, including privileged roles. We need Privileged Role Administrator to assign Contributor role to Supervisor Managed Identity in order to allow a new resource group creation.
  These user roles can be checked on Privileged Identity Management section on Azure portal.
PowerShell minimum required version: 7.3.6.

Set the PowerShell execution policies for Windows computers:

Check the policies with the command Get-ExecutionPolicy -List. The following output will appear:

        Scope ExecutionPolicy
        ----- ---------------
MachinePolicy       Undefined
   UserPolicy       Undefined
      Process       Undefined
  CurrentUser    Unrestricted
 LocalMachine    RemoteSigned

Set the CurrentUser scope to Unrestricted execution policy to allow executing the Pre-deploy.ps1 script. Use this command:
```
Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser
```

Install Azure CLI for Windows (minimum required version 2.53.0).
Certain Azure Resources Providers need to be registered prior to this deployment. These resources are the following:
- Microsoft.Support
- Microsoft.SerialConsole
- Microsoft.Resources
- Microsoft.ResourceGraph
- Microsoft.Portal
- Microsoft.MarketplaceOrdering
- Microsoft.Features
- Microsoft.CostManagement
- Microsoft.Authorization
- Microsoft.ADHybridHealthService
- Microsoft.ContainerService
- Microsoft.Advisor
- Microsoft.ContainerInstance
- Microsoft.AlertsManagement
- Microsoft.App
- Microsoft.Insights
- Microsoft.KeyVault
- Microsoft.ManagedIdentity
- Microsoft.Network
- Microsoft.OperationalInsights
- Microsoft.Search
- Microsoft.SignalRService
- Microsoft.Sql
- Microsoft.Storage
- Microsoft.CognitiveServices
- Microsoft.ContainerRegistry
- Microsoft.Databricks
- Microsoft.DataFactory
- Microsoft.EventGrid
Check for more information in Microsoft Azure Resources Providers site.

Supervisor's deployment¶

The deployment consists of two main steps:

Deployment of Azure Active Directory (AAD) App registrations by executing the PowerShell script: Pre-deploy.ps1.
Deployment of Sidra's Supervisor ARM template by using "Deploy a custom template" feature in Azure Portal.

Step 1. Pre-deployment script¶

A PowerShell script must be run before the Supervisor is deployed. This script will create some necessary resources to start Sidra's installation like:

Managed Identities*:
- Identity with "Directory-readers" privileges
- Identity with "Application-administrator" privileges
With the appRegistrationName assigned on the Pre-deploy.ps1 (recommendable name: <<CompanyName>>.Supervisor.<<Environment>> ), the script will create two Azure Active Directory (AAD) App registrations:
- <<CompanyName>>.Supervisor.<<Environment>>.Manager. This app registration is used to log in to Azure when the DSU is deployed. These values secrets are stored in Supervisor keyVault:
  - AADApplications--Manager--Password
  - AADApplications--Manager--ApplicationId
- <<CompanyName>>.Supervisor.<<Environment>>.IdentityServerAdAccess. This app registration is used by Authentication Service. When this app registration is created, the values secrets are also included in the Supervisor keyVault for authentication proposals. These secrets are:
  - AADApplications--IdentityServer--ApplicationId
  - AADApplications--IdentityServer--Password
  - AAD--0--ClientSecret
Azure Resource Group

*The managed identities will be stored inside the created Azure Resource Group.

Script parameters¶

For the PowerShell script to execute successfully, these are the necessary parameters:

Parameter name	Description	Example
`appRegistrationName`	Name used in the creation of the ADD app registrations	`Sidra.Supervisor.Dev`
`subscriptionId`	The subscription's ID in which the resources are going to be deployed	`000000-0000-0000-0000-000000000000`
`tenantId`	The tenant ID of the used subscription	`000000-0000-0000-0000-000000000000`
`resourceGroupName`	Name of the resource group which will have the created managed identities. This name is going to be used later for Sidra's resource group name	`Sidra.Supervisor.Dev`
`resourceGroupLocation`	Location where the resource group is created	`northeurope`

Execute Command¶

The steps for deploying the script are:

Open a PowerShell 7 cmd.
In the cmd, go to the location where the Pre-deploy.ps1 script is.

Execute this command in the cmd:

Remember to replace the parameter placeholders for the correct value that you prepared in the previous step.

.\Pre-deploy.ps1 -appRegistrationName [appRegistrationName] -tenantId [tenantId] -subscriptionId [subscriptionId] -resourceGroupName [resourceGroupName] -resourceGroupLocation [resourceGroupLocation]

Example:

.\Pre-deploy.ps1 -appRegistrationName "Sidra.Supervisor.Dev" -tenantId "000000-0000-0000-0000-000000000000" -subscriptionId "000000-0000-0000-0000-000000000000" -resourceGroupName "Sidra.Supervisor.Dev" -resourceGroupLocation "northeurope"

During the execution, a webpage from Azure is going to pop up in your default web browser asking you to log in. Log in with a user that satisfies the prerequisites mentioned before.

Now, the command will output some values. This is an example:

AAD applications
----------------------------------------------------------------
App Registration Identity Server Id: 00000000-0000-0000-0000-000000000000
App Registration Manager Id: 00000000-0000-0000-0000-000000000000
----------------------------------------------------------------

From this script output, take note of those parameters values: App Registration Identity Server Id and App Registration Manager Id.

Step 2. Supervisor's ARM template deployment¶

Now that we have executed the pre-deployment script, we are ready to deploy the Supervisor:

Go to Azure Portal and log in using the same user used for the execution of the pre-deployment script.
Go to "Deploy a custom template" feature. You can use the top search bar in Azure Portal to search for it.
Now click in "Build your own template in the editor".
Replace the content on the template with Supervisor's ARM template (Supervisor.json file).
Click "Save" button.

Now, the following parameter's form of the Supervisor's ARM template will be displayed:

Parameter name	Description	Example	Required
`Subscription`	Selects the same subscription in which you executed the pre-deployment script	`Sidra`	Yes
`Resource Group`	Sets the same name used in the pre-deploy script	`Sidra`	Yes
`Region`	Selects the region in which you want to deploy Sidra's resources	`North Europe`	Yes
`Deployment Prefix`	Sets a prefix value for the name of the deployed resources. Maximum length of four characters. Beginning with a numeric character is not allowed	`sds`	Yes
`Environment`	Sets the environment name. Maximum length of four characters	`dev`	Yes
`Supervisor Admin Email`	Sidra's administrator email	`[email protected]`	Yes
`App Registration Identity Server Id`	Sets the value of "app_registration_identity_server/ApplicationId" from the output of the pre-deployment script	`000000-0000-0000-0000-000000000000`	Yes
`App Registration Manager Id`	Sets the value of "manager/ApplicationId" from the output of the pre-deployment script	`000000-0000-0000-0000-000000000000`	Yes
`Company Name`	Sets the company name	`Sidra`	Yes
`VNet Name`	Name of the existing VNet	`SidraVNet`
`VNet Resource Group`	Name of the existing VNet resource group	`Sidra.VNet`
`Database Subnet Name`	Name of the existing database subnet	`SidraDB`
`Container App Subnet Name`	Name of the existing container app subnet	`SidraContainerApp`
`App Subnet Name`	Name of the existing app subnet	`SidraAppSubnet`
`Storage Subnet Name`	Name of the existing storage subnet	`SidraStorage`
`Default Subnet Name`	Name of the default subnet	`SidraDefSubnet`
`Default Dsu Databricks Host Subnet Name`	Name of the default Databricks host subnet	`SidraHost`
`Default Dsu Databricks Cluster Subnet Name`	Name of the default Databricks cluster subnet	`SidraCluster`

The optional parameters are those related with the integration in the Supervisor deployment of an existing VNet from the customer. By default, a new VNet is created with the Supervisor deployment.

Prerequisites for existing VNet integration

In the resources group where the existing VNet is located, verify the following:

The existing VNet is in the same location where Sidra is going to be deployed
The VNet contains the next subnets:

Subnets	Service Endpoints	Subnet Delegation	Network Security Group
container_app	- Microsoft.KeyVault - Microsoft.Sql - Microsoft.Storage	Microsoft.App/environments	-
database	- Microsoft.KeyVault - Microsoft.Sql - Microsoft.Storage	-	-
app	- Microsoft.KeyVault - Microsoft.Web - Microsoft.Storage - Microsoft.CognitiveServices	-	-
Storage	- Microsoft.KeyVault - Microsoft.Storage	-	-
databricks_cluster	- Microsoft.KeyVault - Microsoft.Storage	Microsoft.Databricks/ workspaces	*Example: `sds-t009-VNet-ngs-databricks-cluster`
databricks_host	- Microsoft.KeyVault - Microsoft.Storage	Microsoft.Databricks/ workspaces	*Example: `sds-t009-VNet-ngs-databricks-host`

*For the subnets associated with Network Security Groups and the creation of these NSG, in the same resource group where is the VNet, create the NSG and name it as: <name> + <select region>. Later, select the created NSG and associate it to the subnet.

There must be two NGSs: ngs-databricks-cluster and ngs-databricks-host.
In the case of deploying an additional DSU, the NSGs and the databricks_cluster and databricks_host subnets (Databricks requires only one Databricks for the deployment) must be duplicated. The Storage and Default subnets necessary to create an additional DSU can be shared between DSUs.

For more information about how to deploy an additional DSU, check this documentation.

Click "Review + Create" button.
Azure will start to validate the parameters. If everything is in order, the user will receive a notification stating Validation Passed.
Click "Create" button.
The deployment process will start and you will be redirect to the deployment page overview. Here you can check if the deployment is working well. When everything has finished you will see the message "Your deployment is complete.".
Click to "Go to resource" and you will be redirect to the resource group with all the created resources.

With the steps above, Supervisor's deployment must be completed.

Enjoy your Supervisor's installation.

FAQ¶

I am having an error executing the pre-deployment script. What am I doing wrong?
- If the error message states something like "Forbidden - Authorization_RequestDenied" or "The client <client> with <user-id> does not have authorization or an ABAC condition not fulfilled to perform action", check that the user you are using to execute the script has all the permissions explained in the prerequisites above.
- If the error is something related with one o more of the parameters of the script, check that the values you are using are correct and similar to the examples this document provides.
Azure Portal is displaying the message Validation failed after clicking the "Review + Create" button. What is happening?
- Check that you are filling all the parameters of the form and that the values you are using are correct. Compare them with the examples this document provides.
In the deployment overview page, I am having the message Deployment has failed or the variant Your deployment failed.
- Check which resource has failed from the resource list and read the error message (it must have a red exclamation with a link to "Error details"). Most of the times it will point what is wrong. Likely, the error will be related with one of the values used in the parameter's form. Check that the values used satisfy the conditions explained and are similar to the examples this document provides.

Info