A local LLM is a large language model that runs on your own computer, without sending text to external servers such as those of OpenAI or Anthropic.
In this article, we will set up a local LLM environment on an M1 Max MacBook Pro with 64GB of memory and an M4 Pro Mac mini with 64GB of memory.
The goal is to be able to generate blog article drafts, summaries, rewrites, and social media posts without worrying about external API costs.
What You Can Do with a Local LLM
A local LLM can be used for purposes such as the following:
- Creating blog article outlines
- Drafting body text
- Generating title ideas
- Summarizing text
- Rewriting
- Proofreading
- Writing social media posts
- Creating prompts for image generation
- Classifying and formatting input data
Compared with high-performance external models, local models may fall short in complex reasoning and the accuracy of up-to-date information.
On the other hand, for routine article generation and text processing, even a local model can deliver practical results.
Ollama and MLX
For running a local LLM on an Apple Silicon Mac, this article considers two candidates: Ollama and MLX.
Ollama
Ollama is a runtime environment that makes it relatively easy to set up a local LLM.
Model downloads, launching, and API-based invocation are all handled within a single mechanism.
It is well suited for the following use cases:
- You want to try a local LLM quickly
- You want to switch between multiple models easily
- You want to call it from your own programs over an HTTP API
- You want to use the same setup on a MacBook Pro and a Mac mini
For a setup like the one in this article, where a Python worker processes jobs received from a VPS, we find Ollama's local API easy to work with.
MLX
MLX is a machine learning framework optimized for Apple Silicon.
It takes advantage of the unified memory in Apple Silicon to run LLMs, image generation models, and more.
MLX is better suited than Ollama for fine-grained tuning and custom implementations.
However, building the whole pipeline with MLX from the start increases the amount of model management and API layering you have to provide yourself.
For these reasons, the plan is to use Ollama for the initial setup and consider MLX if performance or level of control proves unsatisfactory.
Model Size and Memory
With local LLMs, the amount of memory required varies depending on the model's parameter count and quantization method.
Generally speaking, larger models may produce higher-quality text and hold more knowledge, but they also require more memory and take longer to generate.
With 64GB of unified memory, you can run many small and mid-sized models.
Keep in mind, however, that macOS and other applications use that same memory.
What matters is not just raw memory usage, but the following conditions:
- Memory pressure
- Swap usage
- Model load time
- Text generation speed
- App responsiveness during generation
- Crashes of the model or worker
For example, even if the Mac mini routinely uses around 45GB for everyday work, not all of that is memory that is needed at any given moment.
macOS frees up the areas being used as cache whenever necessary.
If memory pressure stays in the green during generation, you can likely keep using the machine as is.
If it turns yellow or red, quit memory-hungry applications such as browsers, Docker, or your IDE.
Criteria for the First Model to Try
For initial testing, there is no need to jump straight to the largest models.
Start by choosing based on the following criteria:
- Japanese generation quality above a certain level
- Easily usable from Ollama
- A quantized version is available
- Runs comfortably on a 64GB Mac
- Commercial use terms can be verified
Since model availability and licenses can change, check the official model card at the time of installation.
How to Test Article Generation
To compare the performance of the M1 Max and the M4 Pro, use the same model and the same prompt.
For example, give an instruction like the following:
Please write a beginner-friendly blog article about the pros and cons of running a local LLM on a Mac. Add headings and keep it to about 1,500 characters.
The items to measure are as follows:
- Model load time
- Time until the first characters appear
- Amount of text generated per second
- Total generation time
- Memory usage
- Swap usage
- Machine temperature
- Fan behavior
- Quality of the generated text
Check not only speed but also how natural the text is and whether the facts hold up.
Using a Local LLM as an API
The ultimate goal here is not manual input through a chat interface.
Jobs received from the VPS need to be passed to the local LLM automatically.
The setup looks like this:
VPS
↓
Mac上のPythonワーカー
↓
OllamaのローカルAPI
↓
生成結果
↓
VPSへ返却
If Ollama is used only within the Mac, there is no need to expose its API to the internet.
The Python worker and Ollama communicate within the same Mac, and the only external communication is the HTTPS connection the Python worker makes to the VPS.
Whether to Keep the Model Resident in Memory
If the model is always loaded in memory, text can be generated as soon as a job arrives.
However, keeping the model resident means the Mac's memory is in continuous use.
For this use case, you can choose between the following two operating modes:
Resident mode
Jobs are handled faster.
This works well for periods when article generation happens frequently.
Load-on-demand mode
This saves memory.
This works well for nights with few jobs, or when sharing memory with an image generation model.
Start by verifying operation with load-on-demand mode, and if response speed is unsatisfactory, adjust how long the model stays resident.
Caveats for Local LLMs
Publishing text generated by a local LLM as-is is risky.
The following checks are necessary:
- Whether the facts are correct
- Whether it has invented features or products that do not exist
- Whether the content requires citations or sources
- Whether it repeats the same content
- Whether the Japanese sounds unnatural
- Whether the intended use is permitted under the model's license
For initial operation, rather than publishing automatically, the system will save output as a draft in Payload CMS, and a human will review it before publication.
Summary
With 64GB of memory, both the M1 Max and the M4 Pro are fully capable of serving as environments for running local LLMs.
Start with Ollama to generate article outlines and drafts.
Try the same model on both machines, compare speed, memory usage, and text quality, and then decide on the role of each.
Next time, we will look at ways to generate blog featured images and other visuals locally, using the same two Macs.