Let's recall that you haven't told us - perhaps because you don't know - what this recording says. However, unless he's completely blowing smoke, it's apparently based on something you said and, as such, cannot be defamatory.
Unless you know what the recording says and share it with us...