Multi-State Markov Models for Insurance

Index the discrete states 0, 1, ..., n. At age x the policyholder is in some state and over a small time interval can either stay or jump. Death and policy termination are absorbing states; any state with no outgoing arrows is absorbing.

Two probabilities recur. The transition probability tp_x^{ij} is the probability that a life in state i at age x is in state j at age x+t. The occupancy probability requires the life to remain in state i throughout the interval (0, t], not just to return there at time t. Dickson, Hardy, and Waters use an overbar to mark the second quantity. The distinction matters whenever there is a recovery arrow.

The transition force μ_x^{ij} is the instantaneous rate of jumping from state i to state j at age x. For i ≠ j, μ_x^{ij} is the limit of (tp_x^{ij}) / t as t goes to zero. The diagonal entry of the generator equals the negative of the total exit force from state i, so each row of the generator sums to zero.

In time-homogeneous models the generator does not depend on age. In age-dependent models the generator is a function of x, and the matrix exponential below has to be replaced by a time-ordered exponential that is normally computed numerically.

Collect the transition probabilities in a matrix P(t) with entries tp_x^{ij} for fixed starting age x. Condition on what happens in the small interval (t, t+dt] to obtain the Kolmogorov forward equations: differentiate P with respect to t and pick up one factor of Q on the right.

In the time-homogeneous case the solution is the matrix exponential of tQ. In the age-dependent case the solution is a time-ordered exponential and is normally computed numerically by a small-step recursion on the same identity.

Staying in state i over (0, t] is an exponential-survival calculation against the total exit force from state i. The probability is the negative exponential of the integrated total exit force.

This is the cleanest formula in the multi-state machinery and is the one most often misused. Confusing tp_x^{ii} (allowed to leave and come back) with the occupancy probability (never leaves) is the classic exam mistake on a model with recovery, like the alive-disabled-dead model with positive μ_x^{10}.

States: 0 = healthy, 1 = disabled, 2 = dead. Forces μ^{01} (becoming disabled), μ^{02} (healthy death), μ^{10} (recovery), μ^{12} (disabled death). The permanent disability assumption sets μ^{10} = 0; the temporary disability assumption allows positive μ^{10}.

ALTAM, ILA 201, RET 201, and CFE 101 all use this model. Long-term care and critical illness pricing extend the same machinery by adding states such as chronic illness, multiple ADL deficiencies, or terminal illness.

When all non-zero forces leave a single live state to absorbing decrement states with no recovery, the multi-state model degenerates to a multi-decrement table. The total force of decrement is the sum of the individual decrement forces, and the marginal decrement probability tq_x^{(j)} integrates the j-th force against total survival.

This is the bridge between FAM-level multi-decrement tables and ALTAM-level Markov machinery. Anything that can be written with associated single-decrement and absolute-rate-of-decrement formulas can be re-derived from the generator with all recovery forces set to zero.

Three states 0/1/2 (healthy/disabled/dead). Constant forces μ^{01} = 0.04, μ^{02} = 0.01, μ^{12} = 0.06, no recovery. Total exit force from state 0 is 0.05, and total exit from state 1 is 0.06.

Direct exponentials give 1p_x^{00} = exp(-0.05) = 0.9512 and 1p_x^{11} = exp(-0.06) = 0.9418. For 1p_x^{01} the conditioning identity gives tp_x^{01} = μ^{01} times [exp(-μ^{12} t) − exp(-(μ^{01}+μ^{02}) t)] divided by (μ^{01} + μ^{02} − μ^{12}).

Substituting: 1p_x^{01} = 0.04 × (exp(-0.06) − exp(-0.05)) / (0.05 − 0.06) = 0.04 × (-0.00940) / (-0.01) = 0.0376. By complement 1p_x^{02} = 1 − 0.9512 − 0.0376 = 0.0112. Out of 10,000 healthy lives at age x: about 9,512 still healthy, 376 disabled, and 112 dead after one year.

Same forces, t = 5. Then 5p_x^{00} = exp(-0.25) = 0.7788 and 5p_x^{11} = exp(-0.30) = 0.7408. The closed-form formula gives 5p_x^{01} = 0.04 × (0.7408 − 0.7788) / (-0.01) = 0.152.

After five years, 77.9% of originally-healthy lives are still healthy, 15.2% are currently disabled, and the residual 6.9% have died. The disabled prevalence rises with t even though incidence is constant, because lives accumulate in state 1 faster than disabled mortality removes them.

Disability income contract on a healthy life at age x. Benefit of 1 per unit time paid continuously while in state 1, with no deferred period. Force of interest δ = 0.04 and the same constant transition forces as above.

Expected present value of benefits is the integral of exp(-δt) × tp_x^{01} over t > 0. Using the closed form for tp_x^{01}, EPV(benefits) = 0.04 × (1/(δ + μ^{12}) − 1/(δ + μ^{01} + μ^{02})) / (μ^{01} + μ^{02} − μ^{12}) = 0.04 × (1/0.10 − 1/0.09) / (-0.01) = 0.04 × (10 − 11.111) / (-0.01) = 4.444.

Continuous level premium P payable in state 0 has EPV equal to P × 1/(δ + μ^{01} + μ^{02}) = P × 11.111. The equivalence principle gives P = 4.444 / 11.111 = 0.40. So the continuous annuity-of-one-while-disabled is 4.44 present-value units against an annuity-of-one-while-healthy of 11.11.

First trap: confusing transition probability with occupancy probability when there is recovery. With μ^{10} > 0, tp_x^{00} can exceed exp(-(μ^{01}+μ^{02}) t) because lives can leave state 0 and return. The occupancy probability cannot.

Second trap: treating the generator as upper-triangular when it is not. Adding a recovery arrow turns a triangular generator into a full matrix and changes the eigenvalues and the matrix exponential. The closed forms in the worked examples above hold only because of the permanent-disability assumption.

Third trap: writing the Kolmogorov backward equation when the forward equation is what the problem wants. Forward: differentiate in t with P(t) on the left of Q. Backward: differentiate in starting time with Q on the left of P(t). ALTAM problems almost always use the forward equation.

Standard textbook: Dickson, Hardy, and Waters, Actuarial Mathematics for Life Contingent Risks, 3rd edition, Chapter 8 (multiple state models) and Chapter 9 (Markov multiple state models). The notation tp_x^{ij} and the alive-disabled-dead canonical example are developed there.

Exam-specific: ALTAM topic 1 (survival models for contingent cash flows) and topic 2 (premium and policy valuation for long-term state-dependent coverages). ILA 201 uses these models inside disability income and waiver-of-premium rider pricing. RET 201 uses them for disability-retirement decrement modeling. CFE 101 uses them lightly in the context of insurer risk classification.