Configure the Integration Hub¶

The Integration Hub (IH) in Sidra enables decoupled, event-driven messaging between services, Data Products and external agents. This guide explains how to configure the Integration Hub both server-side and client-side, using Sidra APIs and available SDKs.

DSU Integration Hub¶

Every deployed Data Storage Unit (DSU) in Sidra includes an Integration Hub by default. As part of the deployment, an Azure Service Bus resource (Microsoft.ServiceBus/Namespaces) is created in the resource group. These resources follow a -sbu suffix naming convention for easy identification.

Server-Side Configuration¶

Once the Azure Service Bus namespace is provisioned, it must be configured for use.

Sidra provides the TopicController within its Core API to perform all necessary setup tasks.

Step 1: Create Topic¶

To publish messages, a Topic must be created:

POST /api/topics/{idDataStorageUnit}/{topicKeyName}

Endpoint Details

You can see the Topics endpoint details here

Step 2: Create SAS Authorization Policy¶

A Shared Access Signature (SAS) policy is required to send, listen, or manage the Topic:

POST /api/topics/{idDataStorageUnit}/{topicKeyName}/authpolicies

Endpoint Details

You can see the SAS Authentication Policies endpoint details here

Step 3: Create Subscription¶

To consume messages, a Subscription must be created for the Topic:

POST /api/topics/{idDataStorageUnit}/{topicKeyName}/subscriptions?subscriptionKeyName={subscriptionKeyName}

Endpoint Details

You can see the Subscriptions endpoint details here

Client-Side Setup¶

Integration Hub can be used via:

Sidra SDK
Direct Azure Service Bus SDK

Sidra Integration Hub SDK¶

Use the Sidra.IntegrationHub.Client NuGet package in .NET applications for simplified publishing and consuming of events. It manages SAS token acquisition and integrates with Sidra APIs.

An example solution is provided:
📦 Download example

Components¶

Publisher: Sends messages to a Topic using configured credentials and policies.
Consumer: Subscribes to a Topic and listens for messages, executing registered handlers.

Example Launch Flow¶

Start the Consumer project.
Launch the Publisher to send test events.
The Consumer receives and processes the events.

Info

Replace placeholders in the sample solution with real values for your Sidra environment.

Code Examples¶

Publisher Example

The following is a basic example of a .NET application acting as a Publisher. The IntegrationHubPublisherOptions object is used to configure the connection.

using Microsoft.Extensions.DependencyInjection;
using Sidra.IntegrationHub.Client.Abstractions;
using Sidra.IntegrationHub.Client.IoCC;
using Sidra.IntegrationHub.Client.IoCC.Options;
using Sidra.IntegrationHub.Client.Sample.Common;

class Program
{
    static async Task Main(string[] args)
    {
        await Console.Out.WriteLineAsync("Starting event publisher..");

        var serviceProvider =
            new ServiceCollection()
                .RegisterEventPublisher(
                    sp =>
                    {
                        var serviceBusOptions =
                                new IntegrationHubPublisherOptions
                                {
                                    TopicName = "<MyTopic>", // Name of the existing Topic where consume the message
                                    PolicyName = "<MyPolicy>", // Name of the existing policy name (Must have 'Listen' policy at least, to be able to consume the message)
                                    TokenProviderUri = new Uri("<https://foo-authentication-capp.foo.region.azurecontainerapps.io/connect/token>"), // Authentication container_app url (Identity Server in this case) token endpoint where to request the user's token
                                    ClientId = "<ClientId>", // Core KV secret: ClientSettings--CoreClient--ClientId Identity provider's client identifier. (Client which scope is to request the Sidra Web)
                                    ClientSecret = "<ClientSecret>", // Core KV secret: ClientSettings--CoreClient--ClientSecret Identity provider client's password credential
                                    Scope = Constants.Publisher.Scope, // Scope of the identity provider's client
                                    IntegrationHubApiEndpoint = new Uri("<https://foo-core-bar-api-capp.foo.region.azurecontainerapps.io>"), // URI of the Sidra's Core API
                                    IdDataStorageUnit = <DSUId> // Id of Data Storage Unit where IntegrationHub is deployed
                                };

                        return serviceBusOptions;
                    })
                .BuildServiceProvider();

        var eventBus = serviceProvider.GetService<IEventBus>();
        var iterations = 1;
        for (var i = 0; i < iterations; i++)
        {
            await Console.Out.WriteLineAsync("Publishing event one..");
            await eventBus.Publish(new EventOne(Guid.NewGuid(), "One", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event two..");
            await eventBus.Publish(new EventTwo(Guid.NewGuid(), "Two", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event three..");
            await eventBus.Publish(new EventThree(Guid.NewGuid(), "Three", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event unknown..");
            await eventBus.Publish(new EventUnknown(Guid.NewGuid(), "Unknown", DateTime.UtcNow));
        }

        await Console.Out.WriteLineAsync("Press enter to end publisher.");
        await Console.In.ReadLineAsync();
    }
}

Consumer Example

The following is a basic example of a .NET application acting as a Consumer. Note the use of the IntegrationHubConsumerOptions object during configuration.

using Microsoft.Extensions.DependencyInjection;
using Sidra.IntegrationHub.Client.Contracts.Managers;
using Sidra.IntegrationHub.Client.IoCC;
using Sidra.IntegrationHub.Client.IoCC.Options;
using Sidra.IntegrationHub.Client.Sample.EventConsumer;
using System.Reflection;

class Program
{
    static async Task Main(string[] args)
    {
        await Console.Out.WriteLineAsync("Starting event consumer..");

        var serviceProviderConsumerOne =
            new ServiceCollection()
                .RegisterEventConsumer(
                    sp =>
                    {
                        var serviceBusOptions =
                            new IntegrationHubConsumerOptions
                            {
                                TopicName = "<MyTopic>", // Name of the existing Topic where to publish the message
                                PolicyName = "<MyPolicy>", // Name of the existing policy name (Must have 'Send' policy at least, to be able to publish the message)
                                SubscriptionName = "<SubscriptionName>", /// Name of the existing Topic's subscription to listen the messages
                                TokenProviderUri = new Uri("<https://foo-authentication-capp.foo.region.azurecontainerapps.io/connect/token>"), // Authentication container_app url (Identity Server in this case) token endpoint where to request the user's token
                                ClientId = "<ClientId>", // Core KV secret: ClientSettings--CoreClient--ClientId Identity provider's client identifier. (Client which scope it's to request the Sidra Web)
                                ClientSecret = "<ClientSecret>", // Core KV secret: ClientSettings--CoreClient--ClientSecret Identity provider client's password credential
                                Scope = Constants.Consumer.Scope, // Scope of the identity provider's client
                                IntegrationHubApiEndpoint = new Uri("<https://foo-core-bar-api-capp.foo.region.azurecontainerapps.io>"), // URI of the Sidra's Core API
                                IdDataStorageUnit = <DSUId> // Id of Data Storage Unit where IntegrationHub is deployed
                            };
                        return serviceBusOptions;
                    },
                    typeof(EventOneHandler).GetTypeInfo().Assembly)
                .BuildServiceProvider();

        await Console.Out.WriteLineAsync("Starting consumer one..");
        var eventConsumerOneManager = serviceProviderConsumerOne.GetService<IEventConsumerBusManager>();
        await eventConsumerOneManager.Start();

        await Console.Out.WriteLineAsync("Press enter to end consumers.");
        await Console.In.ReadLineAsync();
        await eventConsumerOneManager.Stop();
    }
}

Azure Service Bus SDK¶

Alternatively, use the native Azure Service Bus SDK. A second example is available:
Download example

This example sends and receives messages using standard Service Bus APIs.

Azure Service Bus Example

The following is a basic example of a .NET application acting as both Publisher and Consumer.

// See https://aka.ms/new-console-template for more information
using System.Text.Json;
using System.Text;
using Azure.Messaging.ServiceBus;
using Microsoft.Azure.Amqp;


// Configuration
string connectionString = "Endpoint=sb://<MyServiceName>.servicebus.windows.net/;SharedAccessKeyName=<MyPolicy>;SharedAccessKey=<ClientId>;EntityPath=<MyTopic>";
string topicName = "<MyTopic>";
string subscriptionName = "<SubscriptionName>";

Console.WriteLine("Publisher:");
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);

// create the sender that we will use to send to our topic
ServiceBusSender sender = client.CreateSender(topicName);

// create a message that we can send. UTF-8 encoding is used when providing a string.
var bodyMessage = "Hello world message 1: " + DateTime.UtcNow;

var message = new ServiceBusMessage
{
    SessionId = "1",
    Body = new BinaryData(bodyMessage)
};

Console.WriteLine("  Sending message: " + bodyMessage + " ------> SessionId: " + message.SessionId);

// send the message
await sender.SendMessageAsync(message);
Console.WriteLine("  Message Sent");

System.Threading.Thread.Sleep(2000);

bodyMessage = "Hello world message 2: " + DateTime.UtcNow;

message = new ServiceBusMessage
{
    SessionId = "1",
    Body = new BinaryData(bodyMessage)
};

Console.WriteLine("  Sending message: " + bodyMessage + " ------> SessionId: " + message.SessionId);

// send the message
await sender.SendMessageAsync(message);
Console.WriteLine("  Message Sent");

Console.WriteLine("");

Console.WriteLine("Consumer:");

// create a receiver for our subscription that we can use to receive the message
ServiceBusReceiver receiver = client.CreateReceiver(topicName, subscriptionName);

// the received message is a different type as it contains some service set properties
ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();

// get the message body as a string
string body = receivedMessage.Body.ToString();
Console.WriteLine("  Received ------> " + body + " ------> SessionId: " + receivedMessage.SessionId);

// the received message is a different type as it contains some service set properties
receivedMessage = await receiver.ReceiveMessageAsync();

// get the message body as a string
body = receivedMessage.Body.ToString();
Console.WriteLine("  Received ------> " + body + " ------> SessionId: " + receivedMessage.SessionId);