MTP on a MoE is hit or miss. If you're bottlenecked on memory, MTP can increase ...

phamilton 20 days ago | parent | context | favorite | on: GLM-5.2 – How to Run Locally

MTP on a MoE is hit or miss. If you're bottlenecked on memory, MTP can increase the number of active experts (like any batch processing would), which can eat away gains from it.