After 5.5 hours on the phone with QBO support (which is a terrible experience because you can't actually talk directly to the person doing the troubleshooting so it takes about 10 times longer than it should), we figured out the problem.

If you don't list a rate for the part number when you purchase inventory, it will not track the COGS properly. The dollar value will stay in inventory.

In Quickbooks desktop software, if you specify the quantity and rate, amount will auto-populate. Likewise, if you specify quantity and amount, rate will auto-populate. This is not the case for QBO, however. I had a huge number of entries where I specified quantity and amount. Those entries created the problem. I believe QBO should ensure that rate is auto-calculated.

I still have an issue with my inventory where part numbers with zero quantity have a non-zero dollar value in the inventory valuation report. Argh, QBO, this is really bad.