Generative AI (GenAI) is a hot topic today and smartphone makers are adding GenAI features as a differentiator. While the chipset powering most of these devices features a dedicated NPU to handle AI tasks, Arm, at MWC 2024, showcased smartphones running GenAI on a Cortex CPU.

The demos showcased Cookie and Ada chatbots on Xiaomi's Redmi K60 Ultra powered by a MediaTek Dimensity 9200+ SoC, running ARM Cortex-A715 CPU core. The app with chatbot Cookie runs Meta's Llama2 7B parameter LLM, whereas the chatbot Ada runs Microsoft's Phi-2 2.7B parameter LLM. The custom apps developed by ARM limit the RAM usage up to 3.5GB, and the CPU can do inferencing at 10 tokens/second. Here's a short demo.

Follow us on social media platforms:
Twitter -   / counterpointtr  
LinkedIn -   / counterpoint-technology-market-research  
Instagram -   / counterpointresearch  

@arm @mediatek @Microsoft @meta @RedmiIndiaOfficial @xiaomi @XiaomiIndiaOfficial