Does Your Data Have Integrity? A Case Study

by the CAS Committee on Management Data and Information

The CAS Committee on Management Data and Information developed the following case study in response to some comments that actuaries were unaware of their responsibilities regarding data. The committee decided a case study giving examples of proper practice would be helpful to practitioners. The committee plans to expand the case study into a presentation at the 2005 Ratemaking Seminar in Atlanta.

Q An actuary has completed his review of a company and is reading the opinion letter one final time before signing. He sees that the opinion letter includes a sentence "I evaluated the data for reasonableness and consistency" and wonders whether or not he has actually done this.

He asks his manager and his manager states, "As long as the year-end case reserves and the paid losses for the most recent calendar year used in your analysis match the totals shown in Schedule P Part 1, you are fine." The actuary is confident the figures reconcile. But is that sufficient?

A This is necessary but not sufficient, for a number of reasons:

1.   Reconciliation to Schedule P should be done in more detail. As explained in the annual Property and Casualty Practice Note for Statements of Actuarial Opinion on P&C Loss Reserves, produced by the Committee on Property and Liability Financial Reporting, the comparison should be done by line of business, by accident year, to the extent such detail was relied upon significantly.

2.   Items other than case reserves and paid losses should also be reconciled. To the extent paid defense and cost containment expenses, incurred defense and cost containment expenses, paid adjusting and other expenses, and earned premium were relied upon significantly in forming the actuarial opinion, these also need to be reconciled to Schedule P, as explained in the Practice Note.

3.   "Evaluation of data for reasonableness and consistency" goes beyond reconciling to Schedule P.

• ASOP 23, Data Quality, describes this review as identifying data values that are questionable or relationships that are materially inconsistent.
• The Practice Note gives one example of something the actuary may choose to investigate—cumulative paid loss amounts that significantly exceed subsequent cumulative paid loss amounts for the same accident year and coverage (unless the actuary is aware of a valid reason for downward developments in the particular circumstances).
• The proposed revision of ASOP 23, currently circulated for comments, states that when reviewing data, if the same work has been done for the prior period the actuary should review the prior period data for consistency with the current period data, and if the actuary does not have the prior period data the actuary should consider requesting it.
• The White Paper on Data Quality, produced by the CAS Committee on Management Data and Information, states that the actuary should determine the extent of checking, verifying, and auditing done by the data manager/supplier and comment on the confidence, reliability, and value of the data quality procedures done by the data manager/supplier.

Little, if any, other guidance is provided, so it is left up to the actuary to determine how to go about this review. ASOP 23 states that the actuary is not expected to develop additional data compilations solely for the purpose of searching for questionable or inconsistent data. Other than reviewing cases where paid losses develop downward and current period data differs from prior period data, as mentioned above, an actuary may want to consider some of the following:

• Are there any individual development factors that appear inconsistent with the remainder of the data?
• Are there instances of unexpected negative case reserves?
• Are there unusual points contained in triangular compilations of ratios of paid-to-incurred losses, loss ratios, reported claims per exposure units, paid or reported claim severity, closed claims to reported claims, or any other data compilation reviewed?

The actuary is by no means compelled to review any of these particular items, they are simply shown here as examples of things an actuary may want to consider. Nor should this be considered an exhaustive list of things to review.

4.   It's not enough to do it; you have to document it. ASOP 23 states the actuary should maintain adequate documentation to support the use of specific data underlying the actuarial work product. This should include any work done to evaluate its reasonableness and consistency. The Practice Note states that the underlying actuarial work papers, including documentation of the reconciliations required by paragraph 10 (i.e., the reconciliation to Schedule P), must be maintained at the company and available for examination for seven years.

Note that it is not necessary to conclude from these tests that the data is "perfect," as perfect data is difficult, if not impossible, to find. Items initially found unreasonable or inconsistent can either be explained or adjusted such that the actuary's analysis is not materially affected. What to do when this cannot be done is beyond the scope of this article.