Good point - but I don't think this is a problem.    The loan put money in the bank in the first place.   You are just changing the character of these funds from a "liability" to "non-taxable income".   It is the same money.