One of the things I've noticed retrospectively is that both Buffy and Angel reach a point when the characters have largely dealt with their initial set of issues and are relatively functional, and then proceed to utterly crush them with new misfortune. For Buffy, that point is late S4 and early S5, until Joyce starts getting ill. (For Angel, it runs from when the others forgive Angel in late S2, until Wes starts falling apart in S3.)