Skip to content

Sidra Data Platform

Data Intake Process

Initializing search

Getting started
Sidra Web
Sidra Core
Sidra Core API
Sidra Data Products
Data Product API
Tutorials
Release Notes

Sidra Data Platform

Getting started
Getting started
Sidra Web
Sidra Web
Sidra Core
Sidra Core
- What is Sidra Core?
- Sidra Resource Groups
- Integration Hub Service
- Orchestration
- Computation
- Model Serving Platform
- Operational DW reports
- Metadata
  Metadata
  - Overview
  - Authorization
    Authorization
    
    Overview
    
    Auth tables
    Auth tables
    
    IdentityAccess table
    
    IntegrationHubAccess table
  - Data Catalog
    Data Catalog
    
    Data Catalog tables
    Data Catalog tables
    
    AttributePopularity table
    
    DataPreviewLoadHistory table
  - Data Ingestion
    Data Ingestion
    
    Overview
    
    Metadata
    Metadata
    
    Metadata Overview
    
    Data Source
    Data Source
    
    Data Source tables
    Data Source tables
    
    DataSource table
    
    DataSourceTemplate table
    
    DataSourceTemplatePipelineTemplate table
    
    Provider
    Provider
    
    Provider tables
    Provider tables
    
    Provider table
    
    ProviderTag Table
    
    Entity
    Entity
    
    Entity Overview
    
    Entity tables
    Entity tables
    
    Entity table
    
    EntityEntity table
    
    EntityDeltaLoad table
    
    EntityPipeline table
    
    EntityTag table
    
    Entity validation error table
    
    Entity actions
    Entity actions
    
    AdditionalProperties configuration
    
    GenerateDeltaTable configuration
    
    Attribute
    Attribute
    
    Attribute tables
    Attribute tables
    
    Attribute table
    
    AttributeTag table
    
    TypeTranslation table
    
    Asset
    Asset
    
    Asset Overview
    
    Asset tables
    Asset tables
    
    Asset table
    
    AssetAdditionalProperties table
    
    AssetIndexingStatus table
    
    AssetPart table
    
    AssetStatus table
    
    Configuration scenarios
    
    Other tables
    Other tables
    
    DataIntakeProcess table
    
    Tag table
    
    TagType table
    
    TableFormat table
    
    DataArchivalRule table
    
    LoadRestrictionMode table
    
    ValidationErrorsChecking table
    
    Azure Data Factory
    Azure Data Factory
    
    Overview
    
    Generic Artifacts
    
    Configure Linked Services - Data Source
    
    Component templates
    
    ADF tables
    ADF tables
    
    Data Factory table
    
    ADF Triggers tables
    ADF Triggers tables
    
    Triggers overview
    
    TriggerTemplate table
    
    TriggerPlatform table
    
    Trigger table
    
    TriggerPipeline table
    
    ADF DatasetTemplate table
    
    ADF ActivityTemplate table
    
    ADF Pipelines tables
    ADF Pipelines tables
    
    ADF Pipelines Overview
    
    PipelineTemplate table
    
    PipelineTemplatePlatform table
    
    PipelineTemplateType table
    
    DatasetTemplatePipelineTemplate table
    
    ActivityTemplatePipelineTemplate table
    
    EntityPipeline table
    
    Pipeline table
    
    PipelineLoadRestriction table
    
    PipelineLoadRestrictionObject table
    
    PipelineLoadRestrictionMode table
    
    PipelineExecution table
    
    Data Storage Unit
    Data Storage Unit
    
    DSU tables
    DSU tables
    
    DataStorageUnit table
    
    ClusterType table
    
    Storage table
    
    StorageRole table
    
    LandingZone table
  - Apps
    Apps
    
    Apps tables
    Apps tables
    
    App table
    
    AppTag table
    
    AppTags table
  - Management
    Management
    
    Management tables
    Management tables
    
    Configuration table
    
    AzureRegion table
    
    ImportExportTask table
  - Notifications
    Notifications
    
    Notification tables
    Notification tables
    
    Notification table
    
    NotificationRead table
- Data Ingestion
  Data Ingestion
  - Data Intake Mechanisms
  - Concepts
    Concepts
    
    Data-lake approach and Data Intake Process
    
    Connector plugin
    
    ADF Ingestion pipelines
    
    Supported Data Sync mechanisms
    
    Table creation
    
    DSU Ingestion script
    
    PII Detection
    
    Encrypted ingestion
  - Data Intake types
    Data Intake types
    
    Overview
    
    Data Intake via landing zone
    Data Intake via landing zone
    
    Configuration steps
    
    Data Intake from complex Excel files
    Data Intake from complex Excel files
    
    Configuration steps
    
    Scenarios
    
    Data Intake with document indexing
    Data Intake with document indexing
    
    Configuration steps
    
    Knowledge store ingestion script
    
    DI Indexing concepts
    
    Data Intake Process via connector plugins
    Data Intake Process via connector plugins
    
    About Sidra connector plugins
    
    Connector plugins versions
    
    Configuration steps
    
    Schema evolution
    
    Data Sync mechanisms for plugins
    
    Sidra connector plugins types
- Deployment
  Deployment
  - Overview
  - Size parameters for Sidra Core
  - Sidra.exe CLI tool
  - Commands
    Commands
    
    Prerequisites command
    
    Profile command
    
    Install command
    
    Deploy command
    
    Provider command
    
    Customize command
Sidra Core API
Sidra Core API
- Overview
- Metadata
  Metadata
  - Data Source
  - Plugin
  - Data Intake Process
  - Provider
  - Entity
  - Asset
  - PII detection
  - Attributes
  - Tags
  - Pipelines
  - Azure Search
- Data Factory
  Data Factory
- Data Storage Unit
  Data Storage Unit
- Management
  Management
- Identity
  Identity
  - Identity Server User
- Ingestion
  Ingestion
  - Inference
  - Ingestion
- Data Querying
  Data Querying
  - Query
Sidra Data Products
Sidra Data Products
- What is Sidra Data Product?
- Concepts
- Data Product Metadata
- Data Product Pipelines
- Data Product Tables
- Size parameters for Sidra Data Products
- Create a new Data Product
- Data Product templates
  Data Product templates
Data Product API
Data Product API
- Overview
- Data Extraction
- Data Factory
- Pipeline
- Sync
- Validation
- Tabular
- Power BI
- Search Services
- Management
Tutorials
Tutorials
- Metadata
  Metadata
- Data Intake
  Data Intake
- Sidra Data Product
  Sidra Data Product
- Management
  Management
Release Notes
Release Notes
- Timeline
- 2023
  2023
  - 2023.R1 Nosy Northern Spy (May 22)
- 2022
  2022
- 2021
  2021
  - 2021.R2 Glamorous Goldspur (Sep 29)
  - 2021.R1 Inquisitive Idared (Apr 27)
- 2020
  2020
- 2019
  2019

Data Intake Process¶

This API's documentation site provides reference information about endpoints of the Data Intake Process.

Last update: 2023-06-22

Made with Material for MkDocs