Introduction

Temporal independence within the context of the Architecture refers to the property of a system's transaction-based data that all days are treated on an equal basis at all times.

One way to think of this is that any processing or queries that can be run for the "current" time period (e.g., "today" or "this month") are also valid for any day/period, past or future.

We will refer to a number of examples from Data Structures, so it may be well to review that page in advance.

Rationale

Why may temporal independence be a desirable property? The reasons include:

• The firm needs to perform the functions listed above.
• The firm needs to perform the queries listed above.
• It more closely models the application events.
• It makes the system much simpler technically: simpler to process, simpler to access, simpler to maintain, and simpler to extend.

Reference Data

We are addressing transaction data only. We are not discussing reference data (e.g., counterparties, securities), which has certain similarities and differences. Two points to start:

• Reference data has no need for reversal records. Keeping records with the new data is all that's required.
• There is no summarization requirement.

Structurally, then, reference data versioning is simpler than it is for transaction data.

Systems differ in the degree to which they support reference data versioning. Data warehouses in particular often incorporate time-varying dimensions explicitly. It is less common in operational systems.

The major question is not how to implement change history, however. The question is the degree to which a transaction report should reflect the reference data as of the transaction's point in time, or whether it should use a different point (usually the current date). If a counterparty has a name change, for instance, a report of transactions will normally show the current name. For an audit report, however, it might be necessary to show all data as of the transaction point in time.

There are many similar situations, such as the change of a symbol on a security, assigning coverage of a security or customer to an account executive, the status of the firm as a market maker in a particular security, or the department to which a trading desk belongs. The reporting requirement in each case may well change from report to report.

It is important to note that these are reporting issues, not database issues. If a record of versions is kept it is possible to support different reporting requirements.

We have highlighted reporting applications, but reference data changes can affect processing as well. If a transaction in a variable-rate bond is corrected, for instance, it is important to pick up the rate as of the settlement date, regardless of the current rate.

Data Stores

Temporal independence imposes requirements on the storage structures for transaction-related data. The requirements are different depending on the level of the data, i.e., whether it is transaction data or the summarization stored in the positions and balances.

From a system point of view, the most basic design requirement is that no updates to the database can be required solely by the passage of time.

Transaction Data

Temporal independence at the transaction level applies not just to securities trades, but to all records in the register. This includes all regular business transactions (e.g., trades, repos), as well as journals, receives and delivers, wires, coupons and dividends, and the like. Temporal independence leads to the following design implications:

• Modifying transactions (i.e., performing corrections) results in a new record, with the new data, being written to the register. The original data must remain unchanged in order to support as-of reporting (i.e., as of a point before the correction was made). This is the versioning mentioned earlier.
• Transaction events, such as cancellations, corrections, and the various forms of settlement, must be handled outside the register entity. If they change the register, there is no way to report the transaction as of a point before the event occurred. In the Data Structures, this is the purpose of the Association entity.
• All register and association rows are stored with explicit dates. There is nothing on the records to indicate "current date".

These implications result in register entries being immutable: once written, they are never changed.

If you review the sample data, there is nothing there that indicates the current date. It would appear to be at least 6/04, but it could easily be 6/05 or some later date.

As mentioned, no updates to the transaction level can be dependent solely on the passage of time (otherwise the transactions won't be treated on an equal basis with transactions already processed).

In particular, this excludes many common examples of end-of-day processing.

• Changing a status flag to represent settlement status: open, pending, failed.
• Moving records to status tables, e.g., to pending or fails tables.

This is one of the ways that temporal independence leads to simpler systems.

Positions and Balances

The situation is not as straightforward for positions and balances because they are derived data. In fact, there are many systems (including most of the better-known PC personal finance software) that have no separately stored position and balance amounts. The summarized amounts shown on the screen or printed on a report are generated at run-time from the transaction level data.

We are not aware of any firms in our target audience that have adopted this approach, though we know some have seriously considered it. The volume of the data, as well as unfamiliarity with managing systems based on this approach, are the reasons for this. This situation will no doubt change. In the meantime, we proceed to consider frameworks which incorporate explicit summarization entities.

We start with a discussion of those aspects of the Positions and Balances component of the Framework that support temporal independence.

• Summarized values are keyed by the explicit date. In our particular Framework, the year and month are part of the key and each day is a separate field on the record.
• There is no inherent limitation on the number of rows that can be stored, or the values of their keys.
• Updates to future periods are handled exactly the same as updates to the current or prior dates. This means that all updates to positions and balances are real-time.
• There is no indication of the current period.

As with the Transaction Data, if you review the sample data, there is no way to determine the "current date" for the positions and balances.

There are many ways to support temporal independence. It is also possible to identify approaches that do not support it. The most common indicator is summary fields with names such as "current_balance", "prior_balance", and the like.

Likewise, as with the Transaction Data, no updates can be dependent simply on the passage of time. Rolling buckets on a summary record (e.g., from "current_balance" to "prior_balance") as part of end-of-day is probably the most commonly encountered such situation. Temporal independence necessarily eliminates the need for such processing.

Processing Control

While we have repeatedly stressed the importance of not tying the data stores to "current date", it is nevertheless extremely useful (and generally required) to store an indication of current date in the system for a number of reasons:

Audit and control	In general, updates to prior periods on the firm's books and records are not allowed (this is discussed in detail in Date Bases). For this reason the term effective date is used to denote the current processing date. It is normally stored in a system parameter record and updated on a daily basis.
Processing defaults	Though updates to the positions and balances are performed in real time for all dates, there do remain end-of-day functions that must be performed. These are primarily accruals, such as trading P&L and interest income/expense, and housekeeping functions (see Processing Rules). Like all functions, they can be run for any day, prior, current, or future. It is extremely useful to be able to schedule them to run using the current effective date.
Report defaults	Though temporal independence supports producing reports on an equal basis for any date, it is extremely useful to be able to ask for reports as of the current effective date.

The Costs

To this point we have looked at the benefits that temporal independence provides, both to the users and in terms of system design and processing. There are, of course, costs associated with temporal independence, and it is important to understand them and whether the benefits outweigh them in your situation.

Design and Development

The largest cost is associated with design and subsequent development. The design is more abstract than other approaches. It can be more difficult to understand and validate.

In order to gain the benefits, the design must be adhered to rigorously. For those unfamiliar with the approach it can be easy to overlook a point that renders all the other work without value. Management of the development process can not be approached in a haphazard manner.

Access

Programming queries against the database is slightly more complex than for other approaches (it is indeed convenient to be able to simply hard-code the field name "current_balance").

While not as involved as the design and development tasks, it will generally require programmers who have greater expertise.

Performance

It might appear that temporal independence imposes performance costs, both in terms of storage requirements and processing time. This is generally minimal, however.

Copyright 2005-2021, Eclipse Software, Inc (ESI).  E-mail: WebSite@eclipsesoftware.biz.