There are too many ways of avoiding UK taxes. Buy-to-let and buying housing for investment distort the housing market and have given some richer individuals the opportunity to become very rich almost TAX-FREE. This activity should be fairly taxed along with others.
Anyone selling a house can avoid 3 years' CGT just by claiming that house as their PPR (Principal Private Residence) for a very short period. So a buy-to-let landlord who is mobile, or has suitable family members to help – can juggle several properties and by judicious moving-in and moving-out can virtually avoid CGT completely. I rent, and it is commonplace to see landlords moving in for a month or two between rentals to activate this over-generous allowance (as well as maybe use the empty space).
My suggestion is to remove it completely – provide an incentive to sell promptly and ensure properties remain occupied as much as possible. If this is too strong, reduce the exemption to, say, a year, and also specify an actual period which counts at all – apparently there is no defined minimum – perhaps 6 months?