Sidra Data Platform (version 2023.12: Quaint Querina)¶

released on Dec 22, 2023

Exciting news from the Sidra Data Platform! Our December release is here, featuring the launch of a new Data Quality Service aimed at ensuring data integrity through an intuitive user experience. Additionally, we've enhanced various functionalities, such as the metadata model, the Sidra Service modularization, or the UI adjustments. Dive in to learn more about these latest improvements!

What's new in Sidra 2023.12¶

New Data Quality Service¶

Sidra now includes Data Quality Service as part of its suite of Services, offering a more intuitive and efficient way to ensure data integrity. Through the Sidra Web, specifically in the Data Catalog section, users can now easily configure data quality rules powered by Great Expectations.

Discover the Data Quality Service in Sidra.

Modularize Sidra Service¶

In this release, we've focused on enhancing the modularization of the Sidra Service, the foundational core of Sidra. This effort has successfully reduced dependencies and significantly boosted overall efficiency.

See documentation of Sidra Service.

Issues Resolved in Sidra 2023.12¶

The following concerns have been resolved to enhance your experience with Sidra:

Adjusted Integration Runtime dropdown in Sidra Web when configuring Data Source avoiding similar values. #4850
Fixed a problem related with missing records about the progress of Services installations in Supervisor. #4959 #5607
Improved Entities table view in Sidra Web for smaller screens. #5167
Fixed an issue where Authentication Service URL in Sidra Web was returning a 404 error. #5168
Updated and enhanced Python packages with shorter, more readable auto-generated method names. #5186
Solved a problem retrieving data from the search box for permissions in Sidra Web. #5312
Fixed DSU counter in Sidra Web dashboard when more that one DSU was deployed. #5320
Configured pass token renewal for micro-frontend avoiding authorization errors. #5348
Corrected Data Catalog section breadcrumb in Sidra Web. #5572
Fixed Providers list view reducing when expanding left-side menu in Sidra Web. #5576
Removed some Base 64 content over environment image in the Sidra UI. #5588
Fixed an issue where Supervisor was not retrieving the last version of the installed Services in Sidra Web. #5606
Configured an uniform positioning of buttons throughout the Data Intake Process and Data Products pages in Sidra UI. #5623
Fixed an issue when accessing to Authentication database after updating the Service. #5656
Corrected configuration for retry when Databricks DEADLINE_EXCEEDED error is raised. #5675
Fixed images from Services in Sidra UI dashboard. #5704
Resolved a loading issue that occurred when Providers or Entities were saved with images. #5706
Adjusted column size to display the whole Roles names in Sidra UI. #5734
Fixed an issue where the rows number count check was failing causing the Data Product pipeline execution fail. #5799
Enhanced the protection of sensitive information in the integration between ARM templates and scripts for Data Products. #5834
Solved a problem when deploying more than one container replica, as authentication became impossible after exceeding ten requests. #5882

Breaking Changes in Sidra 2023.12¶

Metadata update¶

Description¶

As part of this release, several components on the Sidra metadata model have been modified, mainly:

Entity validation errors table has been deleted (the validations are now performed by the Data Quality Service in the DSU ingestion process).

Validation Error Checking table has been deleted.

Some fields definition have been changed in Assets table.

Metadata Attributes fields have been renamed for clearer meaning, avoiding collisions with customer's Attributes.

Required Action¶

Contact our Sidra's Support Team.

Coming soon...¶

Prepare for exciting advancements coming your way soon. We are actively developing innovative features to enhance Anomaly and PII detection, along with intelligent solutions to accelerate the user experience on Sidra Web. These improvements are aimed at optimizing operations in large-scale governed data lakes and refining our connector framework for greater efficiency. Stay tuned for more updates!

We Want Your Feedback!¶

Your thoughts make Sidra better. For suggestions, issues, or questions, please reach out to us at [email protected].