DS isn't better than Sol and Gemini in the benchmarks, but after working with them all for a while now I'm beginning to understand why people find DS _feels_ so much better.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Feels better to me in the sense that it doesn't try to be too clever. When I ask it to do things it actually generates code that a developer would write. That still lets me build software in layers like I do mentally. And it keeps you in control of where abstractions are.
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
> Some people would say that you don't need this at all with the frontier LLMs
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
Huh, doesn't DeepSeek take the longest because it overthinks way too much? Haven't tested this release yet but was very disappointed with past ones because it takes an eternity to do anything because it constantly overthinks and questions itself way too much instead of just starting.
It thinks a bit more, but if the cache reads are free or almost free, and it blasts through 10x faster than GPT Sol/Terra/Luna, it doesn't really matter. Just don't use the max thinking for implementation, go with high or low.
I have to ask what's everyone's issue with the requests to China. We've had 25 years of American big tech basically capture all our data, and use that to make insane profits.
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
I am going back to DeepSeek for writing code. I was trying out using Sol and liking the result, but this should close a lot of that gap. I use Opus/Sol for an implementation plan and code review also use them for grilling. DeepSeek is also running verifications.
I really like the having DeepSeek Flash + Opus 5.5 as advisor/reviewer at the moment. I run through a $200 Claude Code subscription in less than 24 hours now just for a single project, and I hate fiddling with different accounts every day. Now that I get a 200 USD api credits inlcuded with the subscriptions I have rewritten my coding agent workflow completely.
Not sure if this will help you but it helped me. I was using subagents and workflows for lots of long running tasks, and part of that work included running tests. If tests took longer than 5 minutes, the cache expired, and the whole thing was then being processed as uncached input.
I created a skill to better manage agents, including a way to keep the caches valid, and it has decreased my usage considerably. I now struggle to use my 100% each week and I hit claude hard. I have a reset expiring in 5 days and I'm trying to burn down the tokens (68% with 3d 8h left) quickly so I can use my reset before it expires.
Token efficiency just means understanding the way the system works. Best person to help with that... my old friend claude.
After I stopped letting Claude run tests, my token usage went from "gone by Wednesday" to "what can I automate this Saturday so my tokens won't go to waste".
Yeah, I should see if I can create a similar skill. The tests do spend some time to complete.
Claude: Confirmed: 195M of the 212M cache tokens (92%) come from full re-writes after 5+ minute gaps, accounting for 77% of relative cost. Now I'll check the existing wrappers so the skill
builds on them.
Not OP, but I typically ask it to report status of whatever is running; it can run a background task that reports to it every N minutes. With Claude Code in VS Code and a Pro subscription, it seems like the one-hour cache is the default, making it an issue less often.
As a dev who hasn't used LLMs as an embedded part of their work, and uncertain about it all, i'd be curious to know:
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
Not OP, but I used to run out in a few days, for me it was a matter of suboptimal code review practices with many subagents looking at different aspects, often expanded to look at more than the diff.
I swear I have the same question in my head always when I read such statements! I have never once reached 90% that includes everyday usage across multiple repos, artefact generation, device debugging, long running harness work!
I am not saying OP is not telling the truth. I just really want to understand!
I can drain a $200 OAI plan in 2 hours flat working on my 10 projects simultaneously.
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
Not meant as a bad faith response but have you shipped anything? Do you have income from this? Why 10 projects apart from fueling ADHD and "never finishing a project"?
Not OP and while I’m not draining my accounts in a single day/hour, I’m hitting limits with a $200-Claude and $100-OpenAI account.
> Have you shipping anything?
Yes, hundreds of PRs and shipped new features to 3 different platforms (day job and 2 side-business apps/platforms I’ve been developing for many years, pre-LLM). Those new features have gone out to customers and been used.
> Do you have income from this?
Well my day job for one, and I’ve been able to more easily sell to new clients with my side business. Without the changes I’ve shipped I would have been able to land a couple of the clients I’ve signed on recently.
I don’t think OP’s message implied they are never finishing a project, maybe they edited it?
In addition to the 3 income-generating projects, I have a number of other “just for me” projects. Some are mature now and only need occasional development when I want a new feature. Some are in flux as I build them out (like my “software factory”).
At this point I’m limited by compute and tokens more than anything else.
Interesting, coming from a similar situation, I'm interested what workflow you landed on. Opus planning + available to be called for deep thinking + opus reviwer? or?
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
I gave it a kick and got mixed results. For example, searching for a Microsoft product couldn't find its landing page, and searching for a niche open source package landed in the Debian repos for the top hits, but the home page was in the first 10 hits.
Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
I may be missing something so apologies if I'm telling you something you already know but: there are plenty of web search APIs for agents - the Brave one is the one I use: https://brave.com/search/api/
This and (the not quite yet open) Kimi K2.8 are my main coding engines, and I find them incredibly cost efficient. I find Deepseek in particular is really good with Svelte work.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
Probably a week ago someone posted a thread with the title "Why isn't the industry freaking out about DeepSeek 4.1 Flash?"
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion
2. PHASES.md: implementation phases
3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records)
4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM backends, driving the kernels via AudioWorklets and sequencing them from the host language.
There are two power tools at play here:
1. Deepseek Flash 4.1
2. Clojure and ClojureScript
Deepseek is a BEAST. It just goes on and delivers, phase after phase. What it needs: clear instructions - this is covered by the planning process quite well, but I still find it needs some steering from time to time (the right nudget of wisdom dropped into its thought stream at the right moment works wonders)
And Clojure... well. That article PG wrote about LISP (Beating the averages): what I witness now proves all of those claims.
The agent LOVES Clojure. Especially the REPL (not so much the parentheses).
Winning strategy: give it two tools: start_jvm and clojure_eval. Implement everything else it may need - tools to start up HTTP servers, compile ClojureScript, add dependencies, capture a C64 demo, run Playwright via spel, etc. - as lightweight Clojure namespaces which provide APIs to all these functionality. Let these namespaces be thin: the namespace and the functions therein shall have docstrings which tell the agent what the functions do, but the functions themselves shall be lightweight proxies to the real implementations in sub-namespaces. Then point the agent to these top-level API namespaces via your AGENTS.md. Bam, instant discoverability with minimal context usage. It will do EVERYTHING through clojure_eval and the round-trip time will be phenomenal, a perfect match to the raw speed of the agent.
...
Now all I need to figure out is what will I do when all my dreams become manifest in flesh.
First, it's the tok/s and TTFT from providers. Deepseek blazes through work - at quite good quality - and you don't find yourself waiting for your agents to finish.
Second is the openness. Being able to see the CoT and not being randomly blocked for cyber review. You're never wondering if it's stuck or if it will randomly fail. Combined with the great cost value.. My next subscription will probably be Z.ai
Some people would say that you don't need this at all with the frontier llms. But when extending existing software or writing software that fits into an existing ecosystem, I find I still need to offer a lot of guidance. And it's better that the context window isn't flooded with code that was reverted after I looked at it.
Hit the nail on the head. Development methodologies pivoting at the whim of frontier models have been one of the things that have been worrying me as of late.
Wait wut? You do realize that they are two different companies, right?
It reduces the speed and tends to overthink sometimes which is really annoying. For programming, 4.1/high is the bread and butter in my daily work.
If so OpenCode Go looks good and includes DeepSeek models (yet to try myself).
They are showing this new version scoring significantly higher across multiple coding benchmarks, beating Sol 5.6 (6.1 is the latest) across 10/11 benchmarks.
I am going back to DeepSeek for writing code. I was trying out using Sol and liking the result, but this should close a lot of that gap. I use Opus/Sol for an implementation plan and code review also use them for grilling. DeepSeek is also running verifications.
How?
This is how one of my claude code session looks like right now, one prompt from yesterday:
That’s the “how” we’re wondering about.
I created a skill to better manage agents, including a way to keep the caches valid, and it has decreased my usage considerably. I now struggle to use my 100% each week and I hit claude hard. I have a reset expiring in 5 days and I'm trying to burn down the tokens (68% with 3d 8h left) quickly so I can use my reset before it expires.
Token efficiency just means understanding the way the system works. Best person to help with that... my old friend claude.
I think the tests problem is really having an impact for me. Hadn't considered it before so thanks for flagging.
Can you invert things. What’s deterministic and what needs “thinking”?
Can you create a script that runs things chaining together the deterministic parts with calling out to Claude for glue that needs “thinking”.
Claude: Confirmed: 195M of the 212M cache tokens (92%) come from full re-writes after 5+ minute gaps, accounting for 77% of relative cost. Now I'll check the existing wrappers so the skill builds on them.
Awfully new old friend.
Maybe doing bogus small requests to prevent the cache from expiring?
I mean in the API you can set the cache expiration time right? But I guess claude code can't do this
- How long do you reckon it would take a human to do the task?
- What percentage of features/ feature-extensions you add would have been done at all if you didn't have agentic assistance? Is there a tension between llm productivity gains and 'because we can' feature additions?
- Are many of the tasks for throwaway v1 features, exploratory tries?
- How speced out is the work you give it, eg did this task have pages of specs, how long did the spec take to write?
- How deeply will you manually review the work and how much time will it take? Will you read all the code / review at a high level and interrogate with llm / have the llm review and self iterate?
I am not saying OP is not telling the truth. I just really want to understand!
Maybe you're not being ambitious enough with these things? They are at their most fun when you spec out sprawling architecture documents and watch 1+8 astras buzzsaw through it.
I'm only at 260B for the year, though. I'm sure someone who has used more will be along shortly to chime in
E: The other subthread talking about tests is correct. Expensive models should not waste a single second waiting on tests or validation. If you cannot assign a single cheap model to be the test bitch, then just tell them to frontload everything and clean up later as a single wave.
> Have you shipping anything?
Yes, hundreds of PRs and shipped new features to 3 different platforms (day job and 2 side-business apps/platforms I’ve been developing for many years, pre-LLM). Those new features have gone out to customers and been used.
> Do you have income from this?
Well my day job for one, and I’ve been able to more easily sell to new clients with my side business. Without the changes I’ve shipped I would have been able to land a couple of the clients I’ve signed on recently.
I don’t think OP’s message implied they are never finishing a project, maybe they edited it?
In addition to the 3 income-generating projects, I have a number of other “just for me” projects. Some are mature now and only need occasional development when I want a new feature. Some are in flux as I build them out (like my “software factory”).
At this point I’m limited by compute and tokens more than anything else.
i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):
- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant
- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue
- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant
- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.
Worth to mention: none of those findings overlapped.
I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.
The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.
Lovely to see they're doing their own search though, I love how many folk are actively working on search clones at the moment.
Right now I use each of them in their own respective coding harnesses, deepseek-harness and kimi-code, but I do wonder what they are like inside something like opencode.
At the beginning of the month I realized I had more tokens and more capabilities on Opencode at 8$/month via DS 4.1 flash, than I had on a 200$ account in June with the american providers.
Literally both more tokens and capabilities than Opus 4.8/GPT 5.5.
Even if you pay per token, you're still unlikely to outspend american providers.
https://www.reddit.com/r/DeepSeek/comments/1x84wke/deepseek_...
I have been intrigued about Chinese models for a long time, but never tried anything except an early Qwen. My daily driver was Luna, with occasional elevation to Terra or Sol - all on 5.6 as I found the output of 6 jarringly incomprehensible.
I still had 95 USD on my OpenRouter account, so I thought I would give DeepSeek Flash 4.1 a try.
I think now I can understand the person who created that thread.
Here is the setup I settled on, after a lot of refactoring and calibration of the workflow:
I have a "plan" prompt which explains to the agent (Luna) that I want to have a discussion about an idea I have in mind. The agent should use the Socratic method to pull out all the necessary info from me using pointed questions. When the agent feels it has a complete picture, it should elevate to a higher level model (Sol), and ask its review. If Sol has complaints, Luna keeps asking me until the points raised by Sol are clarified. The review rounds with Sol continue until Sol says "ready to implement".
At this point Luna creates four files in a directory I specify:
1. PLAN.md: the implementation plan with all the details that came out from the design discussion 2. PHASES.md: implementation phases 3. DECISIONS.md: design decisions (this serves the same role as ADRs aka. architectural decision records) 4. DIARY.md: append-only log of what happened during implementation
Files 2-4 are written by the implementing agent. Luna only adds headers to explain the purpose of the files.
Once these files are there, I start up a new Pi session, switch to Deepseek 4.1 Flash and then point it to this plan folder.
And then it goes brrrrr... :-)
Here is what I implemented in the past week using this method:
1. A capture/analysis/decompilation pipeline for C64 demos: I ask the agent to start a selected demo through VICE, and say "on" or "off" when I want it to start/stop capture. The capture process uses VICE's builtin trace functionality: the trace log is piped into Clojure where it is saved into an optimized binary format (chunks of trace events). Once I have everything captured, I unleash my analysis pipeline which checks the write patterns of the executed code: what it wrote (VIC, SID, CIA, memory) and when (rasterline, cycle). Then it splits the execution trace into segments by type deduced from the write patterns (decruncher, loader, raster, music) and then decompiles each segment into both traditional ASM files and a s-exp based syntax which roundtrips back into the in-memory representation. Once I have the results of the analysis pipeline, I can tell the agent to analyze any demopart and it can explain how the effects are built down to the tiniest timing detail. As a first test, I asked it to create a clean-room implementation of Fairlight's legendary intro and two days later I had it, with everything in sexp assembly, including the music routine and its data. (The reason it took so long was that this was the first decomp and it provided a lot of feedback to the pipeline development.) Finally I had it write an assembler, linker and packager which creates a PRG from the assembled and linked artifact, then runs it through VICE. Done.
2. A DSP compiler which lets me compile sexp forms of a C-like DSP language into native DSP kernels running on the JVM, JS and WASM. Currently I have about 600 tests which verify that the behavior of the three targets matches that of a reference interpreter written in Clojure. I have a full numeric tower with i32, f32, f64, support for arrays and structs, typical control flow constructs, and a library of DSP building blocks like oscillators, filters, LFOs, delays and a chorus effect. By the time this is ready (end of the weekend?), I will be able to build synthesizers like Synth1, Surge XT or Vital in Clojure/Script, compile the DSP kernels in the browser, choosing between JS or WASM backends, driving the kernels via AudioWorklets and sequencing them from the host language.
There are two power tools at play here:
1. Deepseek Flash 4.1 2. Clojure and ClojureScript
Deepseek is a BEAST. It just goes on and delivers, phase after phase. What it needs: clear instructions - this is covered by the planning process quite well, but I still find it needs some steering from time to time (the right nudget of wisdom dropped into its thought stream at the right moment works wonders)
And Clojure... well. That article PG wrote about LISP (Beating the averages): what I witness now proves all of those claims.
The agent LOVES Clojure. Especially the REPL (not so much the parentheses).
Winning strategy: give it two tools: start_jvm and clojure_eval. Implement everything else it may need - tools to start up HTTP servers, compile ClojureScript, add dependencies, capture a C64 demo, run Playwright via spel, etc. - as lightweight Clojure namespaces which provide APIs to all these functionality. Let these namespaces be thin: the namespace and the functions therein shall have docstrings which tell the agent what the functions do, but the functions themselves shall be lightweight proxies to the real implementations in sub-namespaces. Then point the agent to these top-level API namespaces via your AGENTS.md. Bam, instant discoverability with minimal context usage. It will do EVERYTHING through clojure_eval and the round-trip time will be phenomenal, a perfect match to the raw speed of the agent.
...
Now all I need to figure out is what will I do when all my dreams become manifest in flesh.