You can run a decent model on it, say highly quantized Qwen or Deepseek R1 getting 5-10 tokens/sec output, but it will be nothing in comparison to a commercial offering like Claude, o3 or Gemini. For that you need a datacenter-class GPU going for $50K-100K a pop.
But a small collective running that box, especially spanning timezones, could potentially be a viable alternative or will be soon -- with obv privacy gains too
