Skip to content

Sidra Data Platform (version 2023.11: Plucky Pippin)

released on Nov 21, 2023

Welcome to the latest update of the Sidra Data Platform. Our November release focuses on improving how you use the platform, with a particular emphasis on network configuration, data management, and user interfaces. Here's a summary of what's new, fixed, and what you can look forward to.

Sidra 2023.11 Release Highlights

What's new in Sidra 2023.11

Supervisor Deployment Flexibility

Deploying the Supervisor is now more flexible than ever. Choose to have Sidra set up a new Virtual Network (VNet) for you, or connect to an existing one. Either way, your resources are neatly organized within your network of choice.

Learn more about Supervisor deployment here.

Data Product Configuration via Sidra Web

Configuring a Data Product is significantly faster and easier. With the latest updates to Sidra Web, you can deploy a Data Product and have it up and running in minutes, streamlining your workflow and saving valuable time. The Data Product released called Data Domain allows you have your data available quickly in Databricks and Azure SQL, at all moment secured within your own cloud environment that guarantees consistent protection, security, and unified governance; enhanced through data quality, enriched by PII, ready to be exploited based on your business logic.

Discover the updated Data Domain setup.

Enable deletion of DIP artifacts from Sidra Web

The new interface brings everything to your fingertips, including a much-requested Delete button to remove existing DIPs. This new option enables the ability to remove the artifacts created in Azure Data Factory associated to a specific Data Intake Process and a soft deletion in Sidra (Core).

See how to edit DIPs.

Issues Resolved in Sidra 2023.11

The following issues have been addressed to enhance your experience with Sidra:

  • Fixed an issue that was causing conflicts between web app updates and Azure Data Factory pipeline setups. #1223
  • Made sure our deployment scripts now work with your own domain names. #1234
  • Handled a challenge where we couldn't find the right version of Azure PowerShell needed, so we switched to the latest one. #1238
  • Cleared up an error where old settings in our data processing notebooks were causing errors. #3001
  • Adjusted how we store extra data details to prevent format-related errors. #3229
  • Fixed an issue that sometimes prevented our system from recognizing installed plugins. #3350
  • Configured token renewal to prevent token expiration after one hour of navigation on the Sidra Web. #3701
  • Fixed a problem in schema evolution when synchronizing data from multiples pipeline executions without changes. #3872
  • Corrected a loading error with the notifications display. #3873
  • Improved the process for creating new data intakes, especially when dealing with restricted data types. #3915
  • Made syncing a large amount of data smoother in both SQL and Databricks environments. #4197 #4442
  • Ensured that Databricks clusters shut down properly after data processing tasks are done. #4827
  • Updated the service installation screens to show version details, so you know exactly what you're working with. #4847
  • Solved a problem where previews of your data tables wouldn't create properly due to a driver error. #4854
  • Fixed a problem where Services data was not displayed in the dashboard of Sidra Web. #4918
  • Fixed an endpoint breaking a container after several DIPs executions. #4941
  • Solved a problem where users could not be added in Authorizations service. #4981
  • Configured progress modal window to close when an error happens installing a Service. #4994
  • Fixed a raising error when editing a Provider or saving without any change in Sidra Web. #5303
  • Adjusted the Databricks notebook to create tables for all your data, not just the first batch it gets. #5165
  • Resolved an issue that required restarting the container after deploying Supervisor deployment. #5297
  • Fixed a loading issue on the Data Storage Unit page in Sidra Web that occurred when returning to the main dashboard. #5321
  • Fixed an import issue where using an outdated anomaly detection tool caused the process to fail. #5357
  • Addressed a rare issue where data wouldn't import into Sidra because the system didn't recognize the data format. #5463
  • Adjusted CSV ingestion to work with DSU Ingestion plugin avoiding related errors. #5493

Coming soon...

Get ready for more improvements that are just around the corner. We're working on new features for Data Quality management, and smarter ways to handle Data Intake Processes. Plus, we're expanding Data Domain tools to give you even more insights and control.

We Want Your Feedback!

Your thoughts make Sidra better. For suggestions, issues, or questions, please reach out to us at [email protected].