Uncle is a goldsmith. I once held a decently sized bar of gold (I'd say 30x10x2cm) in my hand and you're absolutely right, it really is weirdly heavy in a way you'd not expect.
If anyone wants to try it, Sovereign Hill in Ballarat, Australia does daily forgings of a gold bar. They melt it down, pour into a mold, let it cool for a minute or two, and then guests can hold it. I've got a picture of my kids almost dropping ~$300k onto the floor.