The purpose of a data request task is to create a historical API request, by date, for information from a source system. Submitting a task triggers our system to make a scheduled request to the source API for the defined period.

For example, you request Amazon Sponsored Advertising data for March 1st, 2nd, 3rd, 4th, and 5th. There are 20 Amazon Sponsored Advertising data feeds. Each API request is made daily for a unique data feed (i.e., Sponsored Product Campaign, Sponsored Product Keywords, Sponsored Brands Campaigns.....). For 5 days and 20 data feeds, this will create 100 API requests (5 days x 20 data feeds).

Go to your Pipelines page in your Openbridge account. Select Request History in the menu options as shown:

Data request tasks are configured on a pipeline and source basis. A task is defined at a day level.

To help you understand Start and End dates, see below;

For example, the current date is April 30. However, you want to create a task to collect data from April 12, 2022, to April 01, 2022. Since April 12 is the most recent date, this is when you want to "start" collecting data. You want to "end" collecting data on April 01.

If you want to collect data for one day, you will set the start and end dates to April 13 and the end date of April 13.

Once submitted, it can take many days to complete the task. Why may it take so long? Prioritization, timing, and limits.

Data request tasks are run as secondary processes. This means it will take longer for this data to show up in your target destination. Why? Data request tasks are optimized as long-running background schedules, carefully requesting prior dates as API capacity permits. This minimizes impacts on existing go-forward daily, hourly, or lookback tasks already running.

The cadence and timing of requests are set by source system API limits, not Openbridge. For example, Amazon Advertising places a hard 60-day limit for historical data. While you may want data for the last two years, Amazon does not supply it. As such, we do not allow you to make requests for data that is not available or permitted by the source API.

In addition to hard limits imposed by APIs, there may be implied limits too. For example, Amazon Selling Partner API may not have a hard limit like Advertising but only allows 60 requests an hour. If you request 13 months of data for 7 report types, this will result in 2,700 reports (7 reports, by day, for 390 days). This can require close to 10,000 API requests for 13 months of data. This can take over 4 weeks to complete, given the 60 requests per hour limit.

The Openbridge system is optimized to work within API limits, maximizing available API capacity. For example, with Amazon Selling Partner API, we must carefully sequence requests to the Amazon APIs to avoid the Amazon system throttling, which would slow any requests and responses by Amazon.

For more information on requests and scheduling, see https://docs.openbridge.com/en/articles/5675188-understanding-data-pipeline-scheduling

Note: Historical syncs are not available during a trial period.