I never understood the "gotta clean cache every time" with CI/CD. I'm sure it makes sense sometimes but you can make a compromise. I worked at two places where I only clean up our c++ caches on the build systems on the weekend. We did that "just in case" caching was hiding a problem, but would only have a "small 1 week" set back at most. We were not on heavy release cycles though so we could afford that. We never had a single problem traced back to the cache or build hiding something. This was for internal company software. Not sure why people are willing to pay the cost for a full rebuild every time if that full rebuild takes a long time (rust or c++). I'm sure there are cases for it though, just that it doesn't have to be done everywhere.
It's not that you have to, it's that you have many different builds that are going to stomp on each other's caches, plus your build services are often ephemeral - especially since I was at a small startup where we wanted to shut systems down overnight to keep the money.
