The aggregation, arrangement and bonding together of a set of components to form respiratory chain complex IV (also known as cytochrome c oxidase), the terminal member of the respiratory chain of the mitochondrion and some aerobic bacteria. Cytochrome c oxidases are multi-subunit enzymes containing from 13 subunits in the mammalian mitochondrial form to 3-4 subunits in the bacterial forms.