Skip to content

Sidra Data Platform (version 2024.02: Serene Santana)

released on Mar 1, 2024

We're thrilled to introduce Sidra 2024.02! Integration Hub now boasts improved robustness and user-friendliness with stricter naming conventions and enhanced validation checks. Data Product setup has been streamlined with simpler terminology and optimized configurations, making data synchronization more intuitive. Alongside, we've resolved a variety of issues to ensure smoother operations. Dive into these updates and experience the enhanced Sidra Data Platform!

Sidra 2024.02 Release Highlights

What's new in Sidra 2024.02

Enhanced Integration Hub

Our latest updates to Integration Hub introduce critical enhancements for robustness and user experience, including adherence to Microsoft's naming conventions for policies, subscriptions, and topics, and validations for the existence of policies and topics before deletion or subscription. We've also streamlined operations by replacing POST with PUT for topic management, ensuring topic names are free of spaces, improving error handling for non-existent DSUs, topics, and duplicate authorization policies.

Find out more about the Integration Hub configuration in Sidra through our guide.

Improved Data Product setup

Data Product section in Sidra Web now simplifies terminology for data synchronization processes. Key changes include renaming Pipeline to Data Sync and Template to Sync Mode, implementing more intuitive names and adding comprehensive documentation for configuration selections. Specifically, for Data Domains, the DSUExtractToDatabricks template has been renamed to Copy Entities to Databricks, accompanied by detailed descriptions. Execution parameters will now only appear for templates that require them, and the Last Assets status section has been removed to provide a more accurate representation of asset statuses across multiple pipelines.

Explore examples of these improvements in Sidra Web documentation.

Enhanced Role Separation in Database Management

Sidra team is introducing a refined access management strategy for database operations. We will maintain the existing sidra user for server administration tasks. Concurrently, we're creating a new user, sidraCore with a role limited to database ownership, ensuring it has the necessary privileges for daily operations without excessive access rights.

Enriched Data Intake Process observability

Sidra is enhancing its Data Intake Process with a focus on improving the reliability and observability of data ingestion. Our new approach ensures that if an Entity fails to load correctly during the intake process, a notification is sent, even if the pipeline finished successfully. We're implementing a system to aggregate and notify about failed Entities within a batch, starting with Azure SQL and SQL Server. This will provide users with comprehensive context and actionable insights to address and resolve issues more efficiently.

Issues Resolved in Sidra 2024.02

These are the issues resolved during this release in Sidra:

  • Adjusted notifications from SignalR on Sidra Web Manager. #37
  • Improved the Sidra Supervisor's response to Sidra deployment failures. #6476
  • Resolved a blocking issue in the Supervisor during Services updates. #6443
  • Fixed an issue in Supervisor during new installations, where the DSU Ingestion process was not being deployed. #6596
  • Solved an issue in the Supervisor's ARM Template script concerning App registrations authentication. #6646
  • Fixed a minor issue related to uppercase deployment prefixes in Data Quality Service. #6662
  • Resolved an issue with additional DSUs during updates from 2023.12 to 2024.01. #6669
  • Solved a minor problem with Azure SQL connector upon container restart. #6682
  • Enhanced the visualization of sensitive data in the general information of a Data Product in Sidra Web. #6712
  • Removed incorrect link when accessing to Data Quality report for the visualization. #6668

Coming soon...

Prepare for the exciting updates ahead! We're dedicated to enhancing the Sidra Web interface, with a special focus on refining the Data Product section for an even more intuitive user experience. Improvements on next release will aim to offer clearer insights and notifications during the Data Intake Process and Data Sync, providing you with comprehensive tracking information for effective issue resolution. Stay tuned for more advancements!

We Want Your Feedback!

Your thoughts make Sidra better. For suggestions, issues, or questions, please reach out to us at [email protected].