had to refund buyer outside ebay

I had an item returned three weeks ago but despite trying to refund the buyer ebay said i cannot as the return was still open, this was underneath the ebay timeline that said it hade been delivered, tried and tried but in the end buyer gave me bank details and i paid him back that way. now ebay is demanding i refund the buyer.