The Qwen team tried to raise the bar one more time with a little August surprise release of the Qwen 3.8 27B open weight model (along with the much larger one). There has been a huge amount of hype around the release comparing it to Opus-level architectural quality and all sorts of other things. Of course, the 27B model wouldn’t be expected to compete, but after messing around with it I think Qwen has made real improvements over its 3.6 release. It’s not all good, though.
Initial Impressions of Qwen 3.8 27B
I have only played with the 27B version, and one of the headline features is the reasoning level built into it, similar to many hosted frontier models. To me, this has been the most frustrating part of adopting this model because it defaults to XHIGH as its reasoning level and that causes a host of issues.
At this level, all it does is think. Like, it will exhaust your context and your time by arguing with itself over the silliest things. I spent a lot of time trying to turn this down, and it was not easy.
However, it does think well. It is impressive that it can come up with very high quality decisions (with time) on very modest hardware (by modern standards). I went back and compared it to both 3.6 models and honestly, I don’t want to go back. I wouldn’t call it Opus-level thinking exactly, and the local experience is just very different than running agents against the cloud. But - it is very useful. I have gotten it to the point where I can ask it to do something non-trivial and switch to planning the next thing while it crunches the problems knowing that it will get it right (eventually - with lots of thinking later).
llama.cpp Run Configuration
When I first downloaded it, I trashed the model file because it was so chatty, but I tried it again and wanted to document exactly what I had to do to make it useful in the real world.
Hardware / Software:
- GPU 1: RTX 5060 Ti 16GB
- GPU 2: RTX 5060 8GB
- Minisforum DEG1 Dock for GPU 2
- Windows 11
- llama.cpp
And my llama.cpp command line:
.\llama-server.exe
-m ../models/Qwen3.8-27B-UD-Q4_K_S.gguf
--mmproj ../models/Qwen3.8-27B-mmproj-F16.gguf
--ctx-size 90000
--no-mmap
--parallel 1
--tensor-split 2.5,1
--n-gpu-layers 99
--cache-type-k q8_0
--cache-type-v q8_0
--flash-attn on
--reasoning-format deepseek
--split-mode layer
--spec-type draft-mtp
--spec-draft-n-max 3
--jinja
--chat-template-file ../models/chat_template.jinja
The secret that finally made it useful and tamed its insane level of thinking was using a custom jinja template to aggressively tell it to think less. Fortunately, someone else had already made one:
Since I just wanted to force a few flags without endless arguments, I added these two lines at the top to bake in some flags:
{%- set reasoning_effort = reasoning_effort | default('medium') %}
{%- set preserve_thinking = preserve_thinking | default(true) %}
Realistic Performance on Consumer Hardware
So, with my setup I can get reasonable performance for local agentic coding:
- 800-900 t/s prefill
- 35-40 t/s generation
With Qwen 3.6 35B mostly loaded into VRAM I could get 1200-1400 t/s prefill and 70-90 t/s generation with a 60K context window. Obviously, I would like that again, but it’s just not realistic to expect a dense model to match that. I rate the performance that I’m getting as somewhat awkwardly slow but not terrible. It gets frustrating - especially with the aggressive thinking - to try to be interactive. So, that means that I have to find other stuff to do while it is thinking away.
My Image Captioning App Experiment
I invented a little project in order to test this model out. I have wanted a little automation on my blog to classify images, write a good caption, crunch EXIF data and add real context to the captions. Here’s a screenshot of the application that this model has written - I haven’t written a single line of code myself. Considering that this is a WPF .NET application with XAML layout files, that is quite non-trivial. XAML confuses the best engineers and I have never really had success with other local models getting it to produce real working code until now without an insane amount of frustration.
The amazing thing about this little experiment is that I am also using Qwen 3.8 27B as the model that generates the captions. This means it has to use the vision projector and have enough knowledge about wildlife and other image objects to be useful. I am surprised about how much image information is actually trained into this model considering it is mainly a coding / reasoning model. Very cool!
Other Useful Tools
Of course, the model itself is one thing, but the ecosystem around it is just as important. My coding rules file now looks like this to address some of the model’s issues:
# Important Information
- This is a windows-based project: ALWAYS use windows line endings and CLI syntax
- Search the web for information if you do not know something
# Tool Usage
- NEVER use CLI or bash commands to read or write files or explore the folder structure
- PREFER the filesystem mcp server tools for file operations
- Use the codebase-memory-mcp server for project context and re-index at the start of each chat
- DO NOT re-read files after planning. I will not change them.
# Refactoring and Large Changes
- For large refactors or file edits CREATE the structure of a file or functions first and then insert the contents in smaller steps
I also spent a lot of time testing these 2 MCP servers which just clicked for this model. These MCP servers seem to give this model a very concise view of the context of the project to keep the context window and reasoning in check. The default tools baked into my agent just did not work well at all.
- filesystem mcp
- codebase memory mcp
Conclusion
It is hard to know how this model compares to different ones in the real world just by the specs and online reviews. If you are interested in local agentic coding, you should definitely try it. It is significantly better than anything before it in my opinion - once you tune it - and it can do real work. It has its quirks like everything else and isn’t going to win any speed races, but in general it is impressive.
And that’s a wrap! Will there be another surprise or two this year in the local-agentic-coding world? I’m 100% sure!