Aquaman, I wouldn't think the shot would have been loose in the shell mixed in with the powder, the friction and compression of the balls on black powder could have caused a premature explosion, in transport or on firing from a gun. Did you flush any powder out too ?? Any yellow sulphur matrix ??
As for your question of a large drill hole 'ruining' the value; actually, some people would probably like to see the shot inside, and like you say, it makes a neat display. Since it is a common 12 lb'er, I don't think it is a problem. It is now an educational piece too. If it were some uncommon shell, I would hesitate from drilling that large a hole. Just my two cents thoughts...