I don't think that the patent should be allowed to be ended early; the fee is paid to keep the patent protection in place.