Skip to main content

Advice and answers from the Openbridge Team

Search results for:Periscope Data

Data Lake Partitioning
We also partition based on the specific upstream data source to properly handle upstream data source… source Partition = The date partition of the data Object = The data object, stored as compressed… Data Lake Partitioning Approach
Data Destination FAQs
Data lakes are the future, and Amazon Redshift Spectrum allows you to query data in your data lake without… Openbridge automates these data pipelines so you can ingest data into your data warehouse cluster code-free… A target is a single data warehouse or data lake that reflects a unique cluster, host, data set, or database
Data Sources FAQs
No, Openbridge is a data pipeline. We do not persist in your data… Your data in our system is transient, which means the time it takes for us to collect data from a source… created and configured a data source and your target data destination
Understanding Data Duplicates
This could lead to the loss of essential data… However, the upstream data source needs to include important information… This data shows that the order ID reflects a single order with five items
Configuring Your Azure Data Lake Storage Data Destination
Step 2: Create Your Data Lake Container… Configuring an Azure Data Lake destination is quick and easy… Next, we need to create a data lake container
Data Catalog: How We Organize and Manage Data In Your Data Lake or Cloud Warehouse
Reduced Risk of Data Loss: With table versioning, no data is lost during schema changes… The result of this data is a table named customer_v1 in your data lake or warehouse… Data Catalog?
Understanding Data Pipeline Scheduling
Data pipeline job schedules are driven by integration behaviors for both sources and destinations… Historical These schedules entail data requests based on a pre-defined offset to align with data… Specific data sources might update past dates with new data, as Amazon updates impression counts for
Understanding Data Source APIs
As such, there are no "live", instantaneous, and continuous data feeds for this data source as it is… APIs Decide What Data Is Available… API Data vs. Reporting User Interfaces
Batch Data Pipeline FAQs
Collection of FAQs to help troubleshoot issues you may have with your batch data pipeline… If your data changes over time, it is best practice to load data for a particular date once… Normally data delivered to the SFTP server will be routed to the target data warehouse
Historical Data Request Tasks
Understanding the process of making requests for data from APIs for prior dates… So, over 5 days, 20 data feeds will generate 100 API requests (5 days multiplied by 20 data feeds)… Different data sources will have different limits