And with MTP (or other speculation techniques) you can ~double that. | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

SlavikCA 16 days ago | parent | context | favorite | on: GLM-5.2 – How to Run Locally

And with MTP (or other speculation techniques) you can ~double that.

phamilton 16 days ago [–]

MTP on a MoE is hit or miss. If you're bottlenecked on memory, MTP can increase the number of active experts (like any batch processing would), which can eat away gains from it.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact