Sidra Data Platform (version 2023.12: Quaint Querina)¶
released on Dec 22, 2023
Exciting news from the Sidra Data Platform! Our December release is here, featuring the launch of a new Data Quality Service aimed at ensuring data integrity through an intuitive user experience. Additionally, we've enhanced various functionalities, such as the metadata model, the Sidra Service modularization, or the UI adjustments. Dive in to learn more about these latest improvements!
Sidra 2023.12 Release Highlights¶
What's new in Sidra 2023.12¶
New Data Quality Service¶
Sidra now includes Data Quality Service as part of its suite of Services, offering a more intuitive and efficient way to ensure data integrity. Through the Sidra Web, specifically in the Data Catalog section, users can now easily configure data quality rules powered by Great Expectations.
Discover the Data Quality Service in Sidra.
Modularize Sidra Service¶
In this release, we've focused on enhancing the modularization of the Sidra Service, the foundational core of Sidra. This effort has successfully reduced dependencies and significantly boosted overall efficiency.
See documentation of Sidra Service.
Issues Resolved in Sidra 2023.12¶
The following concerns have been resolved to enhance your experience with Sidra:
- Adjusted Integration Runtime dropdown in Sidra Web when configuring Data Source avoiding similar values. #4850
- Fixed a problem related with missing records about the progress of Services installations in Supervisor. #4959 #5607
- Improved Entities table view in Sidra Web for smaller screens. #5167
- Fixed an issue where Authentication Service URL in Sidra Web was returning a 404 error. #5168
- Updated and enhanced Python packages with shorter, more readable auto-generated method names. #5186
- Solved a problem retrieving data from the search box for permissions in Sidra Web. #5312
- Fixed DSU counter in Sidra Web dashboard when more that one DSU was deployed. #5320
- Configured pass token renewal for micro-frontend avoiding authorization errors. #5348
- Corrected Data Catalog section breadcrumb in Sidra Web. #5572
- Fixed Providers list view reducing when expanding left-side menu in Sidra Web. #5576
- Removed some Base 64 content over environment image in the Sidra UI. #5588
- Fixed an issue where Supervisor was not retrieving the last version of the installed Services in Sidra Web. #5606
- Configured an uniform positioning of buttons throughout the Data Intake Process and Data Products pages in Sidra UI. #5623
- Fixed an issue when accessing to Authentication database after updating the Service. #5656
- Corrected configuration for retry when Databricks DEADLINE_EXCEEDED error is raised. #5675
- Fixed images from Services in Sidra UI dashboard. #5704
- Resolved a loading issue that occurred when Providers or Entities were saved with images. #5706
- Adjusted column size to display the whole Roles names in Sidra UI. #5734
- Fixed an issue where the rows number count check was failing causing the Data Product pipeline execution fail. #5799
- Enhanced the protection of sensitive information in the integration between ARM templates and scripts for Data Products. #5834
- Solved a problem when deploying more than one container replica, as authentication became impossible after exceeding ten requests. #5882
Breaking Changes in Sidra 2023.12¶
Metadata update¶
Description¶
As part of this release, several components on the Sidra metadata model have been modified, mainly:
- Entity validation errors table has been deleted (the validations are now performed by the Data Quality Service in the DSU ingestion process).
- Validation Error Checking table has been deleted.
- Some fields definition have been changed in Assets table.
- Metadata Attributes fields have been renamed for clearer meaning, avoiding collisions with customer's Attributes.
Required Action¶
Contact our Sidra's Support Team.
Coming soon...¶
Prepare for exciting advancements coming your way soon. We are actively developing innovative features to enhance Anomaly and PII detection, along with intelligent solutions to accelerate the user experience on Sidra Web. These improvements are aimed at optimizing operations in large-scale governed data lakes and refining our connector framework for greater efficiency. Stay tuned for more updates!
We Want Your Feedback!¶
Your thoughts make Sidra better. For suggestions, issues, or questions, please reach out to us at [email protected].