Any process that decreases the frequency, rate or extent of the chemical reactions and pathways resulting in the metabolism of collagen, any of a group of fibrous proteins of very high tensile strength that form the main component of connective tissue in animals.