I have been using AI for coding and technical work for several years now, and being an engineer, there is no escaping the impact the technology has had on my way of working. However, earlier this year I went down a deep rabbit hole to see what it is really like to run local AI workloads on consumer hardware for agentic coding.
Now that it is July 2026, it’s time to check in, and here are my insights for the first half of the year.
Qwen 3.6 (35b and 27b) models are actually good for agentic coding
I had tried the Qwen 2.5, Gemma 4 and Llama 3.1 models that would fit on 8-16GB of VRAM and in all honesty, to me it wasn’t worth running agentic loops locally until Qwen 3.6 35b came out. That was the game changer of the year. Suddenly I could run this model at IQ4_XXS on an 8GB card and have some memory spilling into the system RAM, but it produced sluggish but actually reasonable results. This was the first model where I could write up a small spec for a change I wanted, hit start and let it generally do its thing without tool calling or basic reasoning issues.
I tried the 27b model as well, but with it being a dense model, there is no way it will run faster than 2 t/s on my PC but the results (tool calls, planning, execution and validation) made me decide to go find some more VRAM to give it a real shot.
Between these two models, there are real agentic coding chops if tuned right — and the tooling matters just as much as the model.
The tooling, configurations and setup is just as important as the models
And that leads me to my second insight of the year. When running hosted models like the ones I have access to at work, the contexts are huge, they are running on amazing hardware so 50-70 t/s is just normal with no real prefill lag, and you can get away with being very lazy as both an engineer and an operator. With local agentic coding, everything has to get more precise.
- The tools have to be configured properly and manually - you have to care about the context size you want to run with.
- Configuring model parameters — such as quantization level, KV cache size, parallelism, multi-token prediction, and other properties really does make a significant difference. When you nail the config on consumer hardware, you can get results that you might mistake for frontier models of 12-18 months ago.
- The tooling (like the agent and MCP servers) really make an amazing difference. You want to give the models very efficient prompts with very efficient tool responses and you want it to use the fewest tokens and get its knowledge in the most efficient ways. With hosted models, the breadth of knowledge baked in and the performance can let you brute force your way to solutions much more easily.
What you realistically can and cannot really accomplish on consumer hardware
If you want any success right now with doing agentic coding with local consumer hardware (ie, up to 32GB VRAM), I have found that you just have to be a much better planner. You have to always start the session with a plan and good research. You have to know the codebase better than you might have to using hosted solutions. And you have to be much more precise about what you want.
I think Qwen 3.6 27b at Q4 is like taking a step back by about 12–18 months, but only in some ways. The actual surprise is how small the gap between the two feels to me. If I apply better thought and planning and do the architectural research in pieces, Qwen can execute fairly well.
It has made me a better engineer
This shouldn’t come as a surprise — diving into the ins and outs of new technology and getting true hands-on-keyboard time naturally makes you a better engineer. But like I mentioned earlier, when the models are this small you (as an engineer) just have to be that much more exact to get them to be useful to you.
- You really learn to think about how to describe what you want more clearly.
- You learn to document constantly to add real, meaningful context to your project.
- You start realizing that your architectural decision records are beginning to be recognized and consistently applied.
- You start caring about the context window you are building in the chat.
- You pay attention to what your tools are doing and want to dig under the hood.
Final thoughts
My little experiment in local agentic coding — while definitely not perfect or really ready for the masses — has proven to me that this technology is really on the cusp of shifting away from simply adding more resources toward more intelligently engineered solutions. Looking ahead, the next 12 months should not be dominated by unchecked scaling but by technological optimizations to bring frontier intelligence down into models that can be run much more cost effectively.
On a final note, I added a banner image to the start of this article. It was AI generated using a large cloud model, and it doesn’t fit the article and it doesn’t really make sense. What I did was I gave the model this article and asked it to generate a banner that visualized the concepts. The point being - whether we are using local AI or huge cloud models, these are just tools and we are still responsible for their output.