Not a physicist, but this seems to depend on the physical setting? Seems there are usually metric with physical meaning in continuum mechanics, e.g., elasticity or GR, but not so much if one is working in say geometric mechanics — one can define a Hamiltonian flow on a symplectic manifold without a metric.