Skip to content

Data Catalog

The Sidra Web Data Catalog provides a user-friendly interface to explore and document the data assets ingested into the Sidra Data & AI Platform. From this view, users can browse or search through Providers, Entities, and Attributes, depending on their assigned permissions.

The Data Catalog is built on the Sidra metadata hierarchy model. Refer to that page for a detailed explanation of each metadata element.

Key Capabilities

Sidra Web Data Catalog enables:

  • Browsing of Providers, Entities, and Attributes available in each of the different DSUs of the platform
  • Viewing of high-level statistics such as total data volume and attribute usage
  • Metadata-based search and filtering (e.g., by name, owner, tags)
  • Secure preview of ingested data
  • Enriched documentation using markdown descriptions, images, and tags

Programmatic Access

All metadata in the Data Catalog is available through a secure API, enabling integration with third-party governance tools such as Microsoft's Purview or Alteryx Connect.

Catalog Dashboard

The main Catalog page displays a card-based view of all configured Providers. Users can also toggle to a list view. Each card contains essential Provider information and links to its detail page and associated Entities.

Sidra Web Data Catalog

Each item includes a contextual action menu for navigating deeper into Entity or Attribute details.

Action Menus

The list view supports sorting, layout customization, and quick navigation:

Compact List UI

Sidra leverages Azure AI Search to support fast and scalable metadata search across Providers, Entities, and Attributes.

Search results can be filtered by:

  • Type
  • Tag
  • DSU

Only tags available to the current user are shown, based on platform security.

Search Character Requirement

The global search functionality requires a minimum of three characters to return results. This limitation is in place to improve performance and reduce noise in large catalogs.

Azure Search

Additional filter controls are available to refine search criteria:

Filter UI

Provider Detail Page

This page shows:

  • Metadata: name, description, owner, size, creation date
  • Associated Entities
  • Optional image and tag assignments

A markdown-based editor enables rich documentation and linking to external systems.

Provider Editor - View Mode

Users with edit permissions can also update metadata, images, and tags:

Provider Editor - Edit Mode

Entity Detail Page

Similar to the Provider detail view, this page displays the full metadata of the Entity, including:

  • Short description
  • Documentation field
  • Tags
  • Attributes with usage metrics

The Attribute popularity metric reflects how frequently each attribute is queried by downstream Data Products.

Entity Editor - View Mode

Data Quality Validations

If the Data Quality Service is installed, the Entity detail page also includes a dedicated section for data quality validation results.

Data Quality Service

Learn more about Data Quality

For more on configuring validation rules, refer to the Data Quality overview.

Attribute Detail Page

Displays the full metadata for a selected Attribute, including type, format, and documentation fields.

Data Preview

Each Attribute includes a preview of sample ingested data:

Data Preview

Sensitive Attributes can be masked using Dynamic Data Masking (DDM). Only users with the appropriate roles (e.g., Admin, MaskedDataReader) will see unmasked values. Other users will see masked outputs.

System Attributes are hidden by default but can be revealed using the Show system attributes toggle.

Data Masking Configuration

Sidra Web supports native SQL Server Dynamic Data Masking for controlled data access in the preview interface.

Masking is applied at the Attribute level using the DataMask field in the metadata. Supported masking types include built-in (email, credit card) and custom rules.

Preview data is retrieved from the DataPreview schema, created during the DSU ingestion process.

Example: Custom Masking Rule

To mask all but the first character of a name, configure the DataMask value as:

MASKED WITH (FUNCTION = 'partial(1,"XXXXXXX",0)')

This ensures only authorized users can access sensitive data in the Data Catalog.