How to configure Sidra's Integration Hub (IH)¶

1. Sidra's DSU Integration Hub feature¶

The DSU's Integration Hub feature is enabled by default in Sidra when any DSU is deployed. Once the (default or additional) DSU is deployed, an additional Azure resource of type Microsoft.ServiceBus/Namespaces will be deployed in the resources group. At Sidra the -sbu suffix is used for these resources to be quickly identified.

2. Server-side setting up¶

Once the Azure Service Bus namespace resource has been deployed in our Sidra's DSU installation, it needs to be set up.

Sidra has within its Core API a controller called TopicController. This controller allows end-users to communicate with Azure to perform all the Topics setup actions needed to work with Integration Hub.

2.1 Topic creation¶

To be able to publish messages in the Azure Service Bus resource, we will need to have a Topic where the messages are going to be published. Sidra helps in the creation of the Azure Service Bus Topic by consuming the following endpoint:

[POST] api/topics/{idDataStorageUnit:int:min(1)}/{topicKeyName}

Documentation of this endpoint can be found here.

2.2 Topic authorization¶

To be able to perform different actions on the Topic, we will need to set up a Shared Access Authorization (SAS) Policy. Sidra helps in the creation of the Azure Service Bus Topic's SAS policy by consuming the following endpoint:

[POST] api/topics/{idDataStorageUnit:int:min(1)}/{topicKeyName}/authpolicies

Documentation of this endpoint can be found here.

2.3 Subscription creation¶

To be able to consume the published messages, we will need to have a subscription for the Topic. Sidra helps in the creation of the Azure Service Bus subscription by consuming the following endpoint:

[POST] api/topics/{idDataStorageUnit:int:min(1)}/{topicKeyName}/subscriptions?subscriptionKeyName={subscriptionKeyName}

Documentation of this endpoint can be found here.

3. Client-side set up¶

Once the server-side setup is complete, the Integration Hub can be initiated in several ways:

Through Sidra packages, which internally use Azure Packages, but provide an abstraction that helps in the creation creation and handling of messages.
Through Azure Service Bus, directly launched through Azure packages.

3.1 Sidra's IH¶

Thanks to the Sidra's Integration Hub SDK we can easily take advantage of DSU's Integration Hub feature.

The Sidra.IntegrationHub.Client package can be used on .NET applications to easily Publish and Consume events from Integration Hub.

The package contains two Service Collection extensions that can be used to register a Publisher and a Consumer so it can be resolved by the Dependency Injection.

The package will get a SAS token from the API automatically and will take care of communication with the Service Bus as well.

We provide a sample of how to use Integration Hub using Sidra packages (example attached files, the nuget.config file and dependencies to Sidra.IntegrationHub.Client and Sidra.IntegrationHub.Client.Sample are included to access to Sidra packages). This solution contains two projects, one that works as a Publisher and other that works as a Consumer:

Publisher. This application publishes events to a messaging system using the Sidra Integration Hub. The Publisher project configures the necessary details for authentication and connection, publishing multiple types of events to a specified topic and handling asynchronous operations. The IntegrationHubPublisherOptions object contains the definition of the communication channel, properties to manage the authentication and the DSU to use from the Publisher point of view.
Consumer. This project is responsible for setting up the event consumer using the Sidra Integration Hub. Consumer incorporates handlers to manage the messages from the specified topic and allows the startup and shutdown of the event listening process. The IntegrationHubConsumerOptions object contains the definition of the communication channel, properties to manage the authentication and the DSU to use.

Example¶

Next, the example shows the IH launching the Consumer and Publisher projects, illustrating how it sends and processes the relevant messages.

Launch the Consumer. The application waits until receives a message to be handled.
Launch the Publisher. It begins to send messages.
The consumer is receiving the messages and handle them, in this case, the handler shows information about the message.

Remember to replace the placeholders with the pre-configured values.

A code example of Publisher and Consumer projects is shown below.

Publish example project

A basic example of the Publisher could be a .NET App with the following content. See IntegrationHubPublisherOptions object when being declared.

using Microsoft.Extensions.DependencyInjection;
using Sidra.IntegrationHub.Client.Abstractions;
using Sidra.IntegrationHub.Client.IoCC;
using Sidra.IntegrationHub.Client.IoCC.Options;
using Sidra.IntegrationHub.Client.Sample.Common;

class Program
{
    static async Task Main(string[] args)
    {
        await Console.Out.WriteLineAsync("Starting event publisher..");

        var serviceProvider =
            new ServiceCollection()
                .RegisterEventPublisher(
                    sp =>
                    {
                        var serviceBusOptions =
                                new IntegrationHubPublisherOptions
                                {
                                    TopicName = "<MyTopic>", // Name of the existing Topic where consume the message
                                    PolicyName = "<MyPolicy>", // Name of the existing policy name (Must have 'Listen' policy at least, to be able to consume the message)
                                    TokenProviderUri = new Uri("<https://foo-authentication-capp.foo.region.azurecontainerapps.io/connect/token>"), // Authentication container_app url (Identity Server in this case) token endpoint where to request the user's token
                                    ClientId = "<ClientId>", // Core KV secret: ClientSettings--CoreClient--ClientId Identity provider's client identifier. (Client which scope is to request the Sidra Web)
                                    ClientSecret = "<ClientSecret>", // Core KV secret: ClientSettings--CoreClient--ClientSecret Identity provider client's password credential
                                    Scope = Constants.Publisher.Scope, // Scope of the identity provider's client
                                    IntegrationHubApiEndpoint = new Uri("<https://foo-core-bar-api-capp.foo.region.azurecontainerapps.io>"), // URI of the Sidra's Core API
                                    IdDataStorageUnit = <DSUId> // Id of Data Storage Unit where IntegrationHub is deployed
                                };

                        return serviceBusOptions;
                    })
                .BuildServiceProvider();

        var eventBus = serviceProvider.GetService<IEventBus>();
        var iterations = 1;
        for (var i = 0; i < iterations; i++)
        {
            await Console.Out.WriteLineAsync("Publishing event one..");
            await eventBus.Publish(new EventOne(Guid.NewGuid(), "One", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event two..");
            await eventBus.Publish(new EventTwo(Guid.NewGuid(), "Two", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event three..");
            await eventBus.Publish(new EventThree(Guid.NewGuid(), "Three", DateTime.UtcNow));
            await Console.Out.WriteLineAsync("Publishing event unknown..");
            await eventBus.Publish(new EventUnknown(Guid.NewGuid(), "Unknown", DateTime.UtcNow));
        }

        await Console.Out.WriteLineAsync("Press enter to end publisher.");
        await Console.In.ReadLineAsync();
    }
}

Consumer example project

A basic example of the Consumer could be a .NET App with the following content. See IntegrationHubConsumerOptions object when being declared.

using Microsoft.Extensions.DependencyInjection;
using Sidra.IntegrationHub.Client.Contracts.Managers;
using Sidra.IntegrationHub.Client.IoCC;
using Sidra.IntegrationHub.Client.IoCC.Options;
using Sidra.IntegrationHub.Client.Sample.EventConsumer;
using System.Reflection;

class Program
{
    static async Task Main(string[] args)
    {
        await Console.Out.WriteLineAsync("Starting event consumer..");

        var serviceProviderConsumerOne =
            new ServiceCollection()
                .RegisterEventConsumer(
                    sp =>
                    {
                        var serviceBusOptions =
                            new IntegrationHubConsumerOptions
                            {
                                TopicName = "<MyTopic>", // Name of the existing Topic where to publish the message
                                PolicyName = "<MyPolicy>", // Name of the existing policy name (Must have 'Send' policy at least, to be able to publish the message)
                                SubscriptionName = "<SubscriptionName>", /// Name of the existing Topic's subscription to listen the messages
                                TokenProviderUri = new Uri("<https://foo-authentication-capp.foo.region.azurecontainerapps.io/connect/token>"), // Authentication container_app url (Identity Server in this case) token endpoint where to request the user's token
                                ClientId = "<ClientId>", // Core KV secret: ClientSettings--CoreClient--ClientId Identity provider's client identifier. (Client which scope it's to request the Sidra Web)
                                ClientSecret = "<ClientSecret>", // Core KV secret: ClientSettings--CoreClient--ClientSecret Identity provider client's password credential
                                Scope = Constants.Consumer.Scope, // Scope of the identity provider's client
                                IntegrationHubApiEndpoint = new Uri("<https://foo-core-bar-api-capp.foo.region.azurecontainerapps.io>"), // URI of the Sidra's Core API
                                IdDataStorageUnit = <DSUId> // Id of Data Storage Unit where IntegrationHub is deployed
                            };
                        return serviceBusOptions;
                    },
                    typeof(EventOneHandler).GetTypeInfo().Assembly)
                .BuildServiceProvider();

        await Console.Out.WriteLineAsync("Starting consumer one..");
        var eventConsumerOneManager = serviceProviderConsumerOne.GetService<IEventConsumerBusManager>();
        await eventConsumerOneManager.Start();

        await Console.Out.WriteLineAsync("Press enter to end consumers.");
        await Console.In.ReadLineAsync();
        await eventConsumerOneManager.Stop();
    }
}

3.2. Azure Service Bus¶

Use Integration Hub via Azure Service Bus packages directly. Here, we provide an example of the solution PublisherConsumer (example attached files).

Example¶

By launching the project, Azure Service Bus sends and receives messages between a Publisher and a Consumer using a topic and subscription model.

PublisherConsumer example project

A basic example of the PublisherConsumer could be a .NET App with the following content:

// See https://aka.ms/new-console-template for more information
using System.Text.Json;
using System.Text;
using Azure.Messaging.ServiceBus;
using Microsoft.Azure.Amqp;


// Configuration
string connectionString = "Endpoint=sb://<MyServiceName>.servicebus.windows.net/;SharedAccessKeyName=<MyPolicy>;SharedAccessKey=<ClientId>;EntityPath=<MyTopic>";
string topicName = "<MyTopic>";
string subscriptionName = "<SubscriptionName>";

Console.WriteLine("Publisher:");
// since ServiceBusClient implements IAsyncDisposable we create it with "await using"
await using var client = new ServiceBusClient(connectionString);

// create the sender that we will use to send to our topic
ServiceBusSender sender = client.CreateSender(topicName);

// create a message that we can send. UTF-8 encoding is used when providing a string.
var bodyMessage = "Hello world message 1: " + DateTime.UtcNow;

var message = new ServiceBusMessage
{
    SessionId = "1",
    Body = new BinaryData(bodyMessage)
};

Console.WriteLine("  Sending message: " + bodyMessage + " ------> SessionId: " + message.SessionId);

// send the message
await sender.SendMessageAsync(message);
Console.WriteLine("  Message Sent");

System.Threading.Thread.Sleep(2000);

bodyMessage = "Hello world message 2: " + DateTime.UtcNow;

message = new ServiceBusMessage
{
    SessionId = "1",
    Body = new BinaryData(bodyMessage)
};

Console.WriteLine("  Sending message: " + bodyMessage + " ------> SessionId: " + message.SessionId);

// send the message
await sender.SendMessageAsync(message);
Console.WriteLine("  Message Sent");

Console.WriteLine("");

Console.WriteLine("Consumer:");

// create a receiver for our subscription that we can use to receive the message
ServiceBusReceiver receiver = client.CreateReceiver(topicName, subscriptionName);

// the received message is a different type as it contains some service set properties
ServiceBusReceivedMessage receivedMessage = await receiver.ReceiveMessageAsync();

// get the message body as a string
string body = receivedMessage.Body.ToString();
Console.WriteLine("  Received ------> " + body + " ------> SessionId: " + receivedMessage.SessionId);

// the received message is a different type as it contains some service set properties
receivedMessage = await receiver.ReceiveMessageAsync();

// get the message body as a string
body = receivedMessage.Body.ToString();
Console.WriteLine("  Received ------> " + body + " ------> SessionId: " + receivedMessage.SessionId);