"return the item to inventory"

Customer Credit Memo. List the item and the Amount you agree to accept it back.

"since the item that he is exchanging for is more expensive."

New Invoice..

"Create a new sale order and charge him the difference"

New Sales = New Invoice. Apply the Credit Memo to the open balance; they still owe the difference.