Todays cars still have a very physical steering column and mechanics, not so easy to switch.
My main trouble with right-hand-drive vehicles is that with some models, I put the windscreen wipers on in every intersection. (The turn signal and wiper switches may be either the same way as in left-hand-drive cars, or mirrored.)
I once had rented a (automatic) Nissan for 3 weeks that was left-hand drive, but the controls were mirrored - including the pedals! Weirdest experience ever. I wasn't able to properly operate my own (manual) car for a week.
Seems like it was formerly right-hand drive and the guys who moved it to the left switched the pedals for some reason. I don't believe it came from the factory like this.
Undoubtedly there are safety concerns in the form of "what happens when the power steering breaks?" With a conventional car the steering gets super heavy, but it's still manageable by most people, especially if the car is moving. In a fly by wire vehicle it's literally a "Jesus take the wheel" moment.
My problem used to be that my right hand would smash into the door whenever I wanted to change gears just after switching LHD to RHD. Highly annoying how ingrained such movements are.