Skip to content

Sidra Data Platform (version 2024.04: Unique Upton)

released on April 30, 2024

We're excited to announce the new Sidra Data Platform release! This version brings substantial improvements in performance and stability, alongside a new connector for SAP HANA. Additionally, we've enhanced support for VNet injection on Data Products, building on the feature introduced last month. You'll also notice various UI enhancements and platform fixes.

Sidra 2024.04 Release Highlights

What's new in Sidra 2024.04

SAP HANA Connector

Today we are welcoming a long-awaited addition to our connector gallery: SAP HANA 2.x! This new connector enables seamless integration with the SAP HANA database.

Please note that this is an initial release, and there are two identified limitations:

  • The Array datatype is currently not supported.
  • Differential loads with CDC are not supported, requiring the use of the EntityDeltaLoad mechanism for setting up delta loading of SAP HANA tables.

Give this new connector a try and share your feedback!

Find more about SAP HANA connector in Sidra.

Sidra's Core High Availability

To enhance the reliability and resilience of the Sidra Service, we've made infrastructure changes to support high availability. The service now runs with a minimum of two replicas, scaling up to four replicas.

Stability and Performance Improvements for the Data Intake Pipelines

Over the past few months, we've been hard at work addressing unusual memory allocation patterns in the Data Intake Processes. These patterns hindered performance and, in some cases, led to pipeline failures when processing large data loads (several thousand of Entities).

Our analysis revealed an issue with the job collecting logs from Azure Data Factory to retrieve job execution metrics. To resolve this, we replaced the Hangfire job with a new container in the Sidra Service and changed the logging approach to a much less memory-intensive model. This not only fixed the anomalous consumption but also provided a more resilient architecture moving forward.

Autogenerate Data Product Subnets

In Sidra's previous release, we added support for injecting Data Products into pre-existing VNets. With this new release, we're enhancing this feature by allowing Sidra to automatically create VNets for the Data Products. Users now have the option to either inject into an existing VNet or have Sidra create a new one directly from the UI.

Issues Resolved in Sidra 2024.04

These are the issues resolved during this release in Sidra:

  • Fixed a transient issue which prevents all the steps on a DIP wizard to be shown. #5611
  • Fixed an issue related to the CORS configuration during the DSU deployment. #6965
  • Fixed an issue in the Linked Service of Sidra's internal logs provider that would cause a failure in the pipeline execution. #7084
  • Fixed an issue in the Data Sync list that cause to not display the layout correctly at Data Sync creation. #7085
  • Fixed an issue that caused a failure in the execution of the job used to populate the databases for Sidra Operational Dashboard. #7093
  • Fixed an issue in the Data Sync execution dates where the values shown were not in UTC. #7097
  • Fixed a transient issue that could cause an incomplete deployment of a Data Product. #7125
  • Fixed an issue in the Data Sync edition where the execution parameters defined were not shown. #7147
  • Ensure that when clicking on 'See more' from the Recent log panel in the home page, the results shown are filtered in function of the panel used to see the logs. #7148
  • Fixed some navigation issues in the Data Catalog paging control. #7159
  • Fixed an issue that removed the search results in the Data Catalog search by Provider, Entity or Attribute. #7160
  • Fixed a transient issues that could cause a deployment failure of Sidra's Core. #7187
  • Fixed an issue in which, under certain scenarios, a Data Product could not be injected in a Sidra's VNet that was created externally. #7196
  • Fixed some issue with the pre-visualization of the markdown controls. #7217
  • Fixed an issue in the Data Catalog layout to avoid issues on the right side panel. #7223
  • Fixed an issue where the wrong plugin version was shown during the plugin update process. #7282
  • Ensure that Data Quality service can use Sidra's VNet when it was created externally. #7304
  • Fixed a transient issue that could cause that the plugin that ingest data into the DSU does not install correctly. #7332
  • Fixed some broken links to the documentation in the connector wizards. #7344
  • Fixed an incorrect label for the user name in the Oracle connection details configuration. #7388
  • Fixed an issue where a credential could be shown in a execution log. #7403

Coming soon...

Our team is working hard on new stability and performance updates of Sidra's Core for the next release, as well as new features related to the recently released support for VNet integration on the Data Products. Also, our connector gallery will welcome the addition of a new connector, so stay tuned for more news next month!

We Want Your Feedback!

Your thoughts make Sidra better. For suggestions, issues, or questions, please reach out to us at [email protected].