Put up that sign and then refuse service to blacks, and you're not fine.
Put up that sign, implement the described policy, receive disproportionate numbers of complaints about blacks (because landlords are racist?), enforce the described policy equally irrespective of race, and you are walking a fine line. You could be sued for that and might not be fine.