You have no way of knowing what their reason was for not hiring you. They don't need a good reason to refuse you employment, they just can't do it because of your race, gender, religion, etc.
That certainly was not a smart thing to do though. If it was their reason it would be perfectly valid. Perhaps they saw you as a risk for sexual harassment claims.