Firehose

Filtered to tagged “MoE inference” · clear filters

All PeopleCompaniesPapersPodcastsHacker News

Browse by tag

18 NOV 2026 · Hugging Face

Transformers has implemented expert streaming to offload MoE model weights, achieving speeds of ~14 tok/s, 6% behind Strata's 13.9 tok/s, by streaming only the routed experts and using a bounded cache that groups_mm runs on as is. AI summary