Skip to main content
Data synchronization processes overview
Search
Search Indirect Tax Support Help and Support.

This article applies to:

  • Product: ONESOURCE Determination,ONESOURCE Indirect Tax

Streaming

You can use this synchronization method for applying small UI or API configuration changes to the client environment within minutes.

About streaming

Streaming is a synchronization method for small, immediate changes made in the user interface (UI) or through Configuration APIs. It reflects these delta changes in the client environment within minutes.

Use streaming

The streaming process works like a continuous conversation between the client and the Mothership.
  1. Change Made:
     A user updates the UI or API. Updates can include editing a tax rule, changing a merchant name, or entering a new Exemption Certificate.
  2. Event Logging:
     The Mothership logs this change as a new event with a sequence number.
  3. Polling:
     The client environment automatically polls (contacts) the Mothership at a configurable interval (default: every 5 minutes), asking for any new events that have occurred since its last check.
  4. Applying the Change:
     The Mothership sends back only the new changes (the “delta”). The client receives this small packet of information and applies the update directly to its database.

Key characteristics and limitations

Fast to apply for UI or API Changes:
 This process is efficient for single configuration changes, as only the change itself is transmitted.
When Streaming falls back to a full import:
 In some scenarios, Streaming can't apply a delta change and will instead run a full import of all configuration data. The streaming client then stops processing delta events and triggers a full import using the configured method: SDI or Bulk Import. Streaming falls back to a full import in the following scenarios:
  • A streaming event fails to apply correctly:
     If a streaming event can't be applied, the system begins reconciliation. It clears the affected entities and triggers a full import to restore sync with Mothership.
  • A bulk import/export event is detected (Full Sync event):
     Mothership logs bulk operations, such as full imports or exports, as
    Full Sync
     events. On the next polling cycle, the client views this event. A delta change is not enough, so it triggers a full import.
  • The
    Sync All Configuration
     button is selected:
     Selecting the
    Sync All Configuration
     button writes a full sync event to the Mothership change log. The streaming client detects this on its next polling cycle and initiates a full import.
    note
    This still occurs on the client's regular polling schedule. It doesn’t trigger an immediate or improvised sync.

Streaming can pause

In addition to the previous full fallback scenarios, there are situations where the streaming client will temporarily pause delta processing. This means it holds off on applying further streaming changes and waits until the triggering condition is resolved. Unlike a fallback, a pause doesn’t necessarily trigger a full import on its own. Streaming won’t resume until the condition that caused the pause has been completed. Streaming will pause in the following scenarios:
  • A UI import event occurs:
     A configuration import pauses delta processing until the import is complete. This prevents conflicting or overlapping changes from being applied simultaneously.
  • A content delta or database change occurs:
     When the system detects a content-level or direct database change, the streaming client pauses. It resumes delta sync after the change is processed.
  • The
    Sync All Configuration
     button is selected:
     Selecting this button pauses the streaming. Streaming resumes after the next polling cycle processes the full sync event. No additional delta changes are applied during this time. Delta processing resumes after the full import is complete.
  • An on-demand sync is triggered (
    Review Changes then Sync
     button):
     The
    Review Changes then Sync
     feature in the DA Dashboard holds changes for review. The streaming client pauses delta processing until the administrator verifies the changes and begins the sync. This ensures that only reviewed and approved changes are applied to the client environment.
Configuration Options:
Streaming Mode
 is the
default operational mode
 for the system, by default scheduled to check for changes every 5 minutes. The fallback process it triggers for full sync events is also configurable. By default, it will fall back to the legacy SDI process. If
Bulk Import Mode
 is turned on via a customer overlay, the system intelligently uses the much faster Bulk Import process for these fallbacks instead.
Scheduling Options:
 The following are turned on using the Overlay.
  • Default: Every 5 minutes for near real time delta changes
  • This can be adjusted to longer intervals (10 minutes, 30 minutes, hourly) based on sync frequency requirements. Faster intervals provide more real time updates but increase polling frequency

Key concepts

Term
Description
Mothership
The central master server is where the primary copy of all configuration data is stored for Determination Anywhere. This is the same place where the ONESOURCE Enterprise Cloud Determination is hosted.
Client/Tenant
Customer-specific instance or environment of DA, which syncs data from the Mothership, and performs tax determination.
Prefetch
A server-side background process that runs on the Mothership on a configured schedule. It generates and validates the export file that client environments consume during a Bulk Import or SDI sync. The
Prefetch
 schedule is a key factor in determining the total time for a configuration change to be reflected in the client environment.
Configuration Data
All the settings you can control, including tax rules, rates, entities, custom logic.
Delta Change
A small, incremental change, such as editing a single rule. This is what Streaming is best at.
Full Sync
A complete synchronization of all configuration data. The system uses SDI or Bulk Import to handle this.
Entity Key
A unique combination of fields (for example, name, type) that SDI uses to identify a piece of configuration.

Example use cases

Scenario 1: A Quick Rule Update (Handled by Streaming)
  • Jane, a Tax Manager, notices a custom tax rule needs a minor correction. Jane signs in, edits the custom rule, and saves her change.
  • Behind the scenes: Streaming picks up the change on its next 5-minute cycle. It then creates a cache file by using the prefetch server to serve the client's polling request. Within minutes, Jane's client environment is updated with the corrected rule.
note
If on-demand-sync is turned on, then Jane can review the changes in DA Dashboard before beginning the streaming process using the
Review Changes then sync
 button available in DA Dashboard.
Scenario 2: Moving Configuration from UAT to Production (Handled by SDI or Bulk Import)
  • John, a System Administrator, has finished configuring and testing a new set of rules in the UAT environment. John exports the entire configuration and is ready to import it into Production.
  • The Old Way (SDI):
     John initiates the import. The SDI process begins, running an entity by entity. It takes hours, and during the process, it fails due to a rule ordering conflict. The team will need to investigate and resolve the issue manually before restarting the sync.
  • The New Way (Bulk Import):
     John initiates the import. The Bulk Import process kicks in. It deletes the old configuration and inserts the new set. The entire operation is completed successfully in under 20 minutes, with no errors.

Related Content