The subject of so-called objective derivatives (in Continuum Mechanics) has a long history and is somehow controversial. Several works concern the formulation of the correct mathematical definition of what they are really and try to unify them all into one definition (or one family). In this paper, we show, finally, that all of them correspond in fact to covariant derivatives on the infinite dimensional manifold Met(B) of all Riemannian metrics on the body. Moreover, a natural Leibniz rule, which allows to define an objective derivative on contravariant tensor fields given one on covariant tensor fields and vice versa makes the distinction between those of them who are of "Lie type" or of "co-rotational type" useless. We calculate furthermore, for an exhaustive list of objective derivatives found in the literature, the corresponding covariant derivative on Met(B).