AI-powered laptops have arrived--faster beyond imagination

The first "Copilot+ PCs" have hit the market. They have the new Copilot key, the first substantial change on the Windows keyboard in almost three decades. Like most things these days, their marketing collateral is all about AI (artificial intelligence).

But more importantly, they aren’t powered by Intel or AMD processors. These are Windows on Arm devices – a move that’s been coming for a long time.

Windows on Arm

The journey of Windows on Arm began in the early 2010s, aiming to bring the Windows operating system to devices powered by ARM architecture. ARM-based processors, known for their energy efficiency and performance, were predominantly used in smartphones and tablets. Microsoft recognised the potential of Windows on Arm, expanding Windows for the mobile and low-power computing space, along with ink and touch experiences. (Arm Holdings is the British semiconductor and software design company that designs the processors that implement the ARM architecture family.)

The first significant step was the release of Windows RT, a variant of Windows 8, in 2012, alongside the Surface RT tablet. It was designed to run on ARM-based processors but faced limitations due to its inability to run traditional x86 desktop applications, leading to a lukewarm reception.

In 2017, the company introduced Windows 10 on Arm, which was capable of running traditional applications as well through emulation, leveraging Snapdragon processors from Qualcomm. Further advancements came with the introduction of the Surface Pro X in 2019, powered by a custom Microsoft SQ1 ARM processor. It demonstrated what could be, but still wasn’t quite there.

But what's an AI PC?

Last year at the Intel Innovation, the company’s annual jamboree, CEO Pat Gelsinger proclaimed: “We are ushering in a new age of AI PC.”

These computers come with a dedicated AI processor, or Neural Processing Unit (NPU), specifically designed to take some load off your computer’s CPU (primary processor) and GPU (graphics processor) when running AI-related workloads. While you can game on a laptop without a discrete GPU, having one is better. The same goes for an NPU for AI workloads. NPUs are capable of handling AI workloads – locally on the machine instead of tapping into the cloud – as much as 10,000 times faster than a standard GPU while being more power-efficient. The local inference opens a range of new personalised experiences and helps with stronger data privacy for certain needs.

In the last few months, leading laptop brands like HP, Dell and Lenovo have launched their line-up of AI PCs in India with a focus on AI capabilities. Many industry analysts have pegged 2024 as the first year of growth in the PC market after many years of decline thanks to these AI PCs.

Three-fourths of laptops in 4 years will be AI PCs

Counterpoint Research predicts that 3 out of 4 laptop PCs sold in 2027 will be AI PCs, but with advanced generative AI capabilities. The research firm categorises GenAI laptop PCs into three: Firstly, AI-basic laptops can perform basic AI tasks via multi-core processors and integrated AI acceleration; secondly, AI-capable devices like gaming PCs are high-performance computers that pack in multi-core processors, GPUs ; and , thirdly, AI accelerators. And finally, the AI-advanced laptops are designed and optimised for AI workloads, offering computational power of at least 40 TOPS (Tera Operations per Second, which denotes how many trillions of operations a system can perform per second and is a reference metric to measure and market NPU performance in AI PCs) and powered by dedicated AI hardware such as advanced GPUs, NPUs, or specialized cores.

Intel's latest Core Ultra processors (Meteor Lake) run only 11 TOPS. Apple’s latest MacBooks with M3 processor do 18 TOPS (the company’s latest M4 chip, currently available on iPad Pro only, hits 38 TOPS though) and AMD's latest Ryzen chips too fall below 20 TOPS. These processors will choke on 7-billion-parameter large-language models (LLMs), which are the threshold for generating high-quality generative AI outputs.

Qualcomm arrives in style

Enter Qualcomm with its new Snapdragon X Elite/Plus chips. These hit 45 TOPS, making them the most potent NPUs. Intel too is expected to reach this level later this year with its ‘Lunar Lake’ chips. 

But Microsoft thought it was just the moment to nail the transition to Arm with Copliot+ PCs. Copliot+ PCs are a subset of AI PCs, certified by Microsoft. They bring together the company’s long-standing flirtation with Arm as well as the recent ‘AI everywhere’ pivot.

According to Shruti Bhatia of Microsoft, the launch of Copilot+ PCs marks a new approach for Windows, with a reimagined platform that integrates CPU, GPU, and a high-performance NPU. “This collaboration with Qualcomm signifies a new era for Windows, where AI takes centre stage,” she said.

Two things have led to this moment. In March 2021, Qualcomm acquired Nuvia, a tiny startup founded by three former Apple engineers who led the development of Apple’s A-series chips that are used in iPhones and iPads, leading up to the current M-series for MacBooks. The other is the sudden and meteoric rise of generative AI capabilities, and therefore, the need for AI accelerators.

PCs, empowered

For any PC to flaunt the Copilot+ branding, it needs to pack in a chipset with a dedicated NPU capable of at least 40 TOPS for complex on-device AI-based operations, along with a minimum of 16GB of RAM and 256GB storage. That’s an impressive baseline. The latest MacBook Air wouldn’t qualify, for example.

Since Apple transitioned to its own chips in 2020, the MacBook Air has been able to smoke Arm-powered, as well as Intel-powered, Windows PCs. But the new Copilot+ PCs, including the latest lineup of Surface Pro and Surface Laptop devices from Microsoft, have ushered in a new wave of powerful, battery-efficient Windows laptops with AI-powered experiences. There was a stamp of approval from the industry as well with Acer, Asus, Dell, HP, Lenovo, and Samsung too announcing a portfolio of new Copilot+ PCs.

Fortunately for Microsoft, some of the biggest apps now natively support Arm and major browsers like Edge, Chrome, Firefox, Brave, etc. are also ARM64 native – unlike at the time of the company’s previous tryst with Arm. This reduces the dependency on emulation required for running non-native apps, which too has been drastically improved.

With Qualcomm giving Apple a run for its money when it comes to silicon, Microsoft is feeling confident enough to challenge Apple on its home turf. “We’re going to outperform them,” Microsoft CEO Satya Nadella told The Wall Street Journal recently, even though Microsoft generated less than $5 billion in device revenue in the 12-month period that ended in March, while Apple sold nearly $30 billion worth of Macs over the same period.

`Intel Inside' days are over?

This monumental moment for Windows PCs is driven by Arm, and Qualcomm. The days of the ubiquitous ‘Intel Inside’ sticker may be behind us. According to William Li, senior analyst at Counterpoint Research, PCs based on Arm architecture will grow in popularity and their market share will almost double from 14% to 25% by 2027.

It might be early days for the AI features inside Windows, but Microsoft has readied a new core, in conjunction with Qualcomm’s chip prowess, that could potentially unlock a new wave of AI features and third-party apps. The company is also integrating more than 40 AI models into Windows on Arm to enable developers to build new AI experiences.

With the Arm puzzle seemingly sorted and the extraordinary interest in AI competencies, Copilot+ PCs are an unexpected reset of Windows PCs, a transformation that’s akin to the ‘Internet PCs’ of the 1990s, powered by Windows 95, some three decades ago.

This merger also marks a tectonic shift in the industry where suddenly the leaders are playing catchup – Intel with its chips and Apple with both its silicon as well as AI capabilities. Meanwhile, businesses and end users are trying to validate the innovations in AI and evaluate those for specific use cases and to drive productivity and efficiency.

                                                    (By arrangement with livemint.com)