All British National (Overseas) passport holders have the right of abode in Hong Kong, and it is explicitly stated on every BN(O) passport. The HKID card number of the BN(O) passport holder is printed on the BN(O) passport. Therefore, it would not cause any security problems to thos countries if BN(O)s also enjoy the visa-free access.
British National (Overseas) passport holders enjoy visa-free access to most developed countries, e.g. EU, Canada, New Zealand already.
All BN(O) holders have Hong Kong permanent residency, but some BN(O) holders may not be eligible for an HKSAR passport because they do not have Chinese nationality. Only granting HKSAR passport visa-free access is not fair to those non-Chinese BN(O) holders.
Moreover, some Hong Kong travellers may wish to enjoy British consular protection or be regarded as British subjects when travelling overseas. According to Article 15 of the Universal Declaration of Human Rights, no one shall be denied the right to change his nationality. If Hong Kong people want to be British when entering thos countries, why don't these countries respect their right to choose their nationality?