Eclipse Software, Inc. Temporal Independence

Contents

Key Links

Introduction

Temporal independence within the context of the Architecture refers to the property of a system's transaction-based data that all days are treated on an equal basis at all times.

One way to think of this is that any processing or queries that can be run for the "current" time period (e.g., "today" or "this month") are also valid for any day/period, past or future.

We will refer to a number of examples from Data Structures, so it may be well to review that page in advance.

Processing

Examples of processing that exhibits temporal independence include:

Queries

Example queries include:

Rationale

Why may temporal independence be a desirable property? The reasons include:

Reference Data

We are addressing transaction data only. We are not discussing reference data (e.g., counterparties, securities), which has certain similarities and differences. Two points to start:

Structurally, then, reference data versioning is simpler than it is for transaction data.

Systems differ in the degree to which they support reference data versioning. Data warehouses in particular often incorporate time-varying dimensions explicitly. It is less common in operational systems.

The major question is not how to implement change history, however. The question is the degree to which a transaction report should reflect the reference data as of the transaction's point in time, or whether it should use a different point (usually the current date). If a counterparty has a name change, for instance, a report of transactions will normally show the current name. For an audit report, however, it might be necessary to show all data as of the transaction point in time.

There are many similar situations, such as the change of a symbol on a security, assigning coverage of a security or customer to an account executive, the status of the firm as a market maker in a particular security, or the department to which a trading desk belongs. The reporting requirement in each case may well change from report to report.

It is important to note that these are reporting issues, not database issues. If a record of versions is kept it is possible to support different reporting requirements.

We have highlighted reporting applications, but reference data changes can affect processing as well. If a transaction in a variable-rate bond is corrected, for instance, it is important to pick up the rate as of the settlement date, regardless of the current rate.

Data Stores

Temporal independence imposes requirements on the storage structures for transaction-related data. The requirements are different depending on the level of the data, i.e., whether it is transaction data or the summarization stored in the positions and balances.

From a system point of view, the most basic design requirement is that no updates to the database can be required solely by the passage of time.

Transaction Data

Temporal independence at the transaction level applies not just to securities trades, but to all records in the register. This includes all regular business transactions (e.g., trades, repos), as well as journals, receives and delivers, wires, coupons and dividends, and the like. Temporal independence leads to the following design implications:

These implications result in register entries being immutable: once written, they are never changed.

If you review the sample data, there is nothing there that indicates the current date. It would appear to be at least 6/04, but it could easily be 6/05 or some later date.

As mentioned, no updates to the transaction level can be dependent solely on the passage of time (otherwise the transactions won't be treated on an equal basis with transactions already processed).

In particular, this excludes many common examples of end-of-day processing.

This is one of the ways that temporal independence leads to simpler systems.

Positions and Balances

The situation is not as straightforward for positions and balances because they are derived data. In fact, there are many systems (including most of the better-known PC personal finance software) that have no separately stored position and balance amounts. The summarized amounts shown on the screen or printed on a report are generated at run-time from the transaction level data.

We are not aware of any firms in our target audience that have adopted this approach, though we know some have seriously considered it. The volume of the data, as well as unfamiliarity with managing systems based on this approach, are the reasons for this. This situation will no doubt change. In the meantime, we proceed to consider frameworks which incorporate explicit summarization entities.

We start with a discussion of those aspects of the Positions and Balances component of the Framework that support temporal independence.

As with the Transaction Data, if you review the sample data, there is no way to determine the "current date" for the positions and balances.

There are many ways to support temporal independence. It is also possible to identify approaches that do not support it. The most common indicator is summary fields with names such as "current_balance", "prior_balance", and the like.

Likewise, as with the Transaction Data, no updates can be dependent simply on the passage of time. Rolling buckets on a summary record (e.g., from "current_balance" to "prior_balance") as part of end-of-day is probably the most commonly encountered such situation. Temporal independence necessarily eliminates the need for such processing.

Processing Control

While we have repeatedly stressed the importance of not tying the data stores to "current date", it is nevertheless extremely useful (and generally required) to store an indication of current date in the system for a number of reasons:
Audit and controlIn general, updates to prior periods on the firm's books and records are not allowed (this is discussed in detail in Date Bases). For this reason the term effective date is used to denote the current processing date. It is normally stored in a system parameter record and updated on a daily basis.
Processing defaultsThough updates to the positions and balances are performed in real time for all dates, there do remain end-of-day functions that must be performed. These are primarily accruals, such as trading P&L and interest income/expense, and housekeeping functions (see Processing Rules). Like all functions, they can be run for any day, prior, current, or future. It is extremely useful to be able to schedule them to run using the current effective date.
Report defaultsThough temporal independence supports producing reports on an equal basis for any date, it is extremely useful to be able to ask for reports as of the current effective date.

The Costs

To this point we have looked at the benefits that temporal independence provides, both to the users and in terms of system design and processing. There are, of course, costs associated with temporal independence, and it is important to understand them and whether the benefits outweigh them in your situation.

Design and Development

The largest cost is associated with design and subsequent development. The design is more abstract than other approaches. It can be more difficult to understand and validate.

In order to gain the benefits, the design must be adhered to rigorously. For those unfamiliar with the approach it can be easy to overlook a point that renders all the other work without value. Management of the development process can not be approached in a haphazard manner.

Access

Programming queries against the database is slightly more complex than for other approaches (it is indeed convenient to be able to simply hard-code the field name "current_balance").

While not as involved as the design and development tasks, it will generally require programmers who have greater expertise.

Performance

It might appear that temporal independence imposes performance costs, both in terms of storage requirements and processing time. This is generally minimal, however.

Storage

It is fairly common to find versioning of trades, often with the reversals. The incremental cost for full support of temporal independence is minimal.

The increased storage for positions and balances is quite a bit larger, however. Production experience with this approach shows that the positions and balances are typically about 10% of the size of the transaction data. It is a business question as to whether the increase is worthwhile.

Processing

As updates to the positions and balances are done on a transaction-by-transaction basis in real-time, it might appear that this will slow the transaction entry process. Experience has shown, however, that the impact is generally unnoticeable. This is true even when a file of transactions is being batch processed. It is certainly more than offset by the savings of eliminating complex end-of-day update routines that can easily become a bottleneck.

Transaction queries can also require more processing. Determining fail status of a certain point of time is an example. Again, however, production experience has shown that the impact is negligible.


Last updated: August 16, 2016.   Copyright 2005-2017, Eclipse Software, Inc (ESI).  E-mail: WebSite@eclipsesoftware.biz.