I personally think votes should only affect the order of comments. I am interested in reading down voted comments just to avoid the echo chamber effect.