For those using Quickbooks Desktop for Mac I have had this problem on multiple occasions and its the result of file corruption. That's what the support steps above aim to address. Unfortunately they don't always work and I have had to delete and recreate invoices in order to correctly apply payments so that there is no lingering customer balance or overpayment.

I believe that on least one occasion this is the result of payments being applied to customers labeled in the system as "inactive." Good luck.