Consider the following scenario: there is a source table for "Product" named product_v1. Openbridge automatically generates an aggregate view for all source tables called product_master . To support downstream ETL or analytics efforts you create a custom view called my_product_view that is based on the product_master­  view.

A few weeks after my_product_view  there is an upstream change to Product that changes to the underlying data structures. In the Openbridge system, this triggers the automated creation of  a new table version to adjust to the change. In this example, a new source table called product_v2 is created . In addition to the new table version, the system will also automatically update product_master view to aggregate two source tables; the older product_v1  and the new product_v2 .

This creates a challenge for the custom my_product_view. Databases systems like Amazon Redshift or Google BigQuery will require that you first drop all associated views before creating a new one. This means that in order to replace product_master views must be dropped. In-place substitution is not permitted by those systems. However, once product_master­­­ is dropped, all other reliant views, including my_product_view, will be dropped as well. This is where a break can occur for custom views like my_product_view. While the Openbridge system automates change management, more often than not a customer is not backing up their custom views or if they are, they are not able to restore a view prior to something breaking. The loss of my_product_view can disrupt ETL, analytics or other data engineering activities.

Our solution is to automatically create a snapshot of all of your views. How does it work? 

We will regularly backup the SQL used to create all views on the database you activated with Openbridge. This process will run once a day at 10 AM UTC. In the event there is a migration that needs to occur, our auto migration process will verify that all affected views have backups available. In most cases, this will be true since we are automatically backing up the table and view definitions. 

However,  if we detect that a backup is missing, the migration process will halt and we’ll be notified that some views are not backed up. We will then fix this for you. If backups are available then the auto-migration will continue we will restore your custom views after the process is completed. 

Our backup may not match your current custom view if it changed after our last backup. For example, we backup  my_product_view  at 2 AM EST.  You make changes to  my_product_view  at 9 AM EST. An auto-migration is triggered by changes to source data at 1 PM EST.  We drop my_product_view as part of the auto-migration and restore with our backup. However, the backup reflects my_product_view as of 2 AM EST, not 9 AM EST. This means we will restore an older version of your view. 

Normally views are slowly changing, so the risk of this use case is low.  However, we recommend that you keep backups of your work as a fail-safe outside of the automated process we undertake on your behalf. In the future, the backup process will be automatically triggered prior to an auto-migration