If the government loses money by collecting taxes, it completely defeats the point of tax. By not collecting these tiny amounts, money would be saved on postage, paper, ink etc. and labour costs for admin. Not much admittedly, but if this situation is as common as it seems to me, that would all add up.
The government should cancel tax owed if it will cost more to collect than the actual amount being paid. For example, I know people who have had to pay amounts as small as 17p in tax, and in order to pay it have had to exchange letters and cheques etc. with the government – easily a couple of pounds of government money spent. If these taxes were not collected the government would actually be better off.