Skip to content

How to add a new Provider

A Provider is a Sidra element that represents a logical collection of Entities.

One Provider can be related to a unique data source or several data sources, which together conform a logical unit.

Providers are stored in the Provider table in the Sidra Core metadata database.

There are two ways to create a new Provider entry in Sidra:

  • Preferred: Create a new Provider by using the Sidra API endpoint;
  • Not recommended: Create a new Provider with a SQL script, to insert a new Provider entry in the Sidra Core database.

IMPORTANT

Some fields of the Provider will be linked to Azure resources; naming conventions and restrictions will apply. Therefore, using the Sidra API to create a new provider is preferred, since the API will perform the needed validation checks, redusing the chances for errors.

Add a new Provider by using the Sidra API

For illustrative purposes, we will assume that the Sidra API is deployed at the following URL; the host-name follows the Sidra's naming conventions:

https://sidr-core-dev-wst-api.azurewebsites.net

Sidra API requires requests to be authenticated: the section How to use Sidra API explains how to create an authenticated requests.
The API comes with the Swagger UI as a test ground for manual calls to the API. Swagger will ease the process authenticating the calls:

https://sidr-core-dev-wst-api.azurewebsites.net/swagger

This is the sequence of requests required to create a new Provider:

Step 1. Get the identifier of the Data Storage Unit

To create a Provider, it is required to know the ID of the Data Storage Unit (DSU). The DSU will hold the data of the Entities associated with the Provider.
If the DSU identifier is already known, this step can be skipped. If not, the following request will retrieve all the DSUs:

Request

GET https://sidr-core-dev-wst-api.azurewebsites.net/api/datastorageunit/datastorageunits?api-version=1.0

It will return an object with the list of DSUs, including their IDs.

Response

{
    "TotalItems": "{TOTAL NUMBER OF DSUs}",
    "Items": [{
            "id": 1,
            "name": "MyDSU",
            "resourceGroupName": "{RESOURCE GROUP NAME}",
            "clusterName": "{CLUSTER NAME}",
            "idClusterType": "{ID CLUSTER TYPE}",
            "idLocation": "{LOCATION ID}",
        }
    ]
}

From the previous response, the ID of the DSU can be retrived and used in the next step to populate the field idDataStorageUnit.

Step 2. Create a new Provider entry

To create a Provider entry, you will need information that needs to be gathered: see below details about all the fields of a Provider entry.

By using Sidra API, some of the fields of the Provider are automatically populated, like Id, ParentSecurityPath or CreationDate.
Once the rest of the information of the Provider is gathered, its entry can be created using this request.

IMPORTANT

For the DatabaseName field, lower letters, figures and spaces are allowed; no underscore (_) or minus (-) may be accepted.

Request

POST https://sidr-core-dev-wst-api.azurewebsites.net/api/metadata/providers?api-version=1.0

And adding the request payload content as required fileds of a Provider entry:

Request body

{
    "providerName": "MyNewProvider",
    "databaseName": "DwMyNewProvider",
    "owner": "John Doe - jdoe@email.com",
    "description": "Description of MyNewProvider",
    "idDataStorageUnit": 1
}

The response will return the ID of the Provider created, which can be used, for example, when adding a new Entity.

Response

{
    "id": 10,
    "itemId": "6C12DF27-2C38-4B0B-9ACD-DFEC2930ACDA",
    "tags": [{
            "name": "string"
        }
    ],
    "providerSize": 0,
    "dataStorageUnitId": 1,
    "creationDate": "2020-01-22T18:03:45.938Z",
    "owner": {
        "name": "John Doe - jdoe@email.com",
        "picture": "string"
    },
    "providerName": "MyNewProvider",
    "databaseName": "dw_newprovider",
    "description": "Description of MyNewProvider",
    "idDataStorageUnit": 1
}

Add a new Provider by using an SQL script

IMPORTANT

This method of adding a Provider entry is not recommended, since no validation checks will be performed on the fields. Adding fields in a wrong format may result in errors in the subsequent steps, such as when adding a Data Source for the provider.

A sample of the script can be found below.

It is recommended that the SQL script should be added - following the naming conventions - to the Scripts\CoreContext folder or to the place configured in the DatabaseBuilder (from where it retrieves the scripts). This would ensure that the Provider entry will be re-created when a new Sidra deployment pipeline run takes place.

-- DECLARE
DECLARE @Id_Provider INT = 
(
    SELECT COALESCE(MAX([Id]) + 1, 1)
    FROM [DataIngestion].[Provider]     
)
DECLARE @Id_Dsu INT = 
(
    SELECT TOP 1 [Id] 
    FROM [DataIngestion].[DataStorageUnit] 
    WHERE [Name] = '<Name of the DSU where the entities of the provider will be stored>'
)

-- ROLLBACK
DELETE FROM [DataIngestion].[Provider]
WHERE [Id] = @Id_Provider

-- SCRIPT
SET IDENTITY_INSERT [DataIngestion].[Provider] ON 
INSERT [DataIngestion].[Provider] (
    [Id],
    [ProviderName], 
    [DatabaseName], 
    [Description], 
    [Owner], 
    [IdDataStorageUnit],
    [ParentSecurityPath],
    [CreationDate]) 
VALUES (
    @Id_Provider,
    N'MyNewProvider', 
    N'DwMyNewProvider', 
    N'Description of MyNewProvider', 
    N'John Doe - jdoe@email.com', 
    @Id_Dsu,
    @Id_Dsu,
    GETUTCDATE())
SET IDENTITY_INSERT [DataIngestion].[Provider] OFF

Provider fields

This is the information about the Providers that must be included when a new Provider is stored in the metadata database:

Column Description
Id [Required] Provider identifier.
ItemId [Required] Provider GUID
ProviderName [Required] Name of the Provider. Spaces are not allowed and it is usually written in camel-case; e.g. "MyNewProvider".
DatabaseName [Required] Name of the database in the metastore of the Data Storage Unit (DSU). Lower letters, figures and spaces are allowed; no underscore (_) or minus (-) may be accepted. An example: "DwMyNewProvider"; where "dw" stands for Data Warehouse.
Description [Optional] Description of the Provider.
Owner [Optional] Identification of the person responsible of the Provide; it can be a name, an email, etc.
IdDataStorageUnit [Required] Id of the DSU -previously called DataLake- where the Entities of the Provider will be stored.
ParentSecurityPath [Optional] The security path of the parent following the metadata model hierarchy.
Image [Optional] Image of the Provider to be displayed in the Data Catalog.
SecurityPath [Computed] The security path of the Provider.
CreationDate [Required] Date of creation of the Provider.
Detail [Optional] Detailed description (markdown allowed) of the Provider to be displayed in the Data Catalog.1

  1. Updated 2022-01-22. 


Last update: 2022-02-10
Back to top