This paper develops a new series of large language models (Xing 4) by optimizing architecture and training infrastructure for efficient use on specialized chips (Ascend). Practitioners may care about the techniques described here because they could improve the performance and scalability of their own AI models.
Firehose
Filtered to tagged “Model optimization” · clear filters
Browse: People · Companies · Papers · Podcasts · Hacker News · Deep dives
Browse by tag
artificial intelligence 63open-weight models 19agentic coding 17continual learning 16AI 15AI agents 11reinforcement learning 9datasette 8AI safety 7cybersecurity 7Databricks 7natural language processing 7AI Gateway 6large language models 6Reinforcement learning 6vision-language models 6computer vision 5Diffusion models 5language models 5machine learning 5open-source 5Agentic AI 4AI ethics 4benchmarks 4education 4productivity 4recursive self-improvement 4Recursive self-improvement 4space 4tech 4
In this episode, Philip Kiely and Ali Taha from Baseten discuss the complexities and innovations in inference engineering for large AI models. They cover topics including model deployment, speculative decoding, quantization, hardware optimi…
Inference engineeringSpeculative decodingQuantizationModel deploymentTool callingKV cacheTensor parallelismExpert parallelismGPU hardwareRubin GPUVideo diffusionAutoregressive modelsDiffusion modelsTraining-inference convergenceContinual learningOpen source modelsInference infrastructureModel optimizationLatency vs throughputMulti-modal models