My weekly ritual for the last month or so has been to head over to the Apple website, configure my dream Macbook Pro, stare at the price for a few minutes before deciding not to pull the trigger. I recently went a step further and hit ‘buy’ only to discover that the delivery window was slap-bang in the middle of my vacation. Once again, I backed down and grumbled about the astonishing cost; a touch over £5,000.
Yesterday, Apple announced price hikes that means that the machine I am considering is now priced at £7,000. Gulp!
The machine in question is intended to replace my 6-year old Windows Desktop and is a 16 inch MacBook Pro with 128GB RAM, 2TB SSD storage and an M5 Max processor with 18 CPUs and 40 GPUs.
Why not a Mac Studio? Surely that would make much more sense as a desktop replacement? That’s because Mac Studios currently max out at an M3 Ultra and 96GB RAM (They used to go up to 512GB but not right now).
It’s the RAM I want and, if I’m being honest, I’d rather have more than 128GB. 512GB would probably make me feel comfortable enough to pull the trigger on a machine that I expect to last for at least the next half a decade.
I am not alone in my desires; everyone wants the RAM and this is the problem.
Local LLMS: The reason why I want so much memory
I want a machine with this much memory because I am currently experimenting with local AI models where memory, and memory bandwidth, are arguably more important than raw compute power. Without a lot of memory that can be directly addressed by the GPUs (often called unified memory), the LLM that drives the AI simply cannot run.
This is a capability issue. If you want to run the AI on your machine and not the cloud, you need a lot of memory. No memory, No AI.
If you thought I was crazy when I said I’d ideally want 512GB RAM to keep me going for the next five years, check out this video by Alex Ziskind where he hooks together 4 Mac Studios, giving a total of 2 Terabytes RAM to run the current cutting edge local AI model.
2 Terabytes! Of RAM! On his desk!
2TB of RAM is the kind of memory I’m used to getting on the ‘High Memory Nodes’ of HPC centers. You know! The special ones that they only have 2 of in a 500-node cluster and you have to speak really nicely to the sysadmin to get access to. Here’s a real world example, this is a screenshot of the specs of The University of Sheffield’s current HPC system. They have 12 ‘Very large nodes’ with 2 TB memory each. Nice!
RAMageddon: Why all consumer electronics is becoming more expensive
The fact that consumers like me are seriously considering machines with so much memory that they wouldn’t look out of place in a HPC datacenter gives a hint as to what’s going on worldwide.
AI needs memory and the world is currently going crazy for AI. The big players in the market have not only bought most of the RAM, they’ve already bought most of the RAM that is going to be produced for the foreseeable future.
As a result, the price of everything that needs RAM: Computers, Phones, games consoles etc is going up. By a lot! This sucks for anyone who is in the market for new tech right now. Since I am not only in the market for a new computer but also a new phone, this sucks a lot for me.
So what am I going to do?
I was hesitating to buy a new computer when it ‘only’ cost ~£5,000 so £7,000 (Around $9,250 for my stateside friends) feels like a non-starter. However, the crazy thing is that some people are predicting that things will get much worse and stay worse until 2030 or so. It may well be the case that I’ll be kicking myself for not taking up this ‘bargain’ in a few months time.
For now, I’ve decided that I’ll take that risk. I’ve kicked myself before; it wasn’t so bad! They say necessity is the mother of invention and right now my necessity is to do more with less. Many people will be in the same boat as me. They’ve recognized that local AI is going to be useful but they don’t have sufficient funds to significantly upgrade their existing kit.
We need to MacGyver the hell out of this. So that’s the plan. For now.