In early years of hash function design it was unclear how to choose constants (not only initial vectors), and it was widely assumed that the more random they look, the more secure the function is. There is still not much research in this direction. However, there have been several attacks (rotational cryptanalysis, slide attacks, internal difference attacks) that exploited similarity between the constants and their low Hamming weight. It may happen that some choice of constants yields a weakness or even a backdoor that allows a faster collision/preimage search.
Recent attacks on Keccak/SHA-3 would have been slower if Keccak state were initialized with some weird constant, not zero as it is now. This is a property of the Keccak permutation, whereas the encompassing sponge construction is invulnerable to the initial vector choice.
Another reason is to choose distinct IVs to differ functions that are identical, but truncate the output to a different number of bits (like SHA-224 and SHA-256).