Hacker News

> they need to see/learn by themselves

The cleanup pay is mighty generous as well.

If only LLM benchmarks could benchmark it in the first day!

Still no Artificial Analysis benchmark yet. Or benchmark for Laguna S 2.1 or Meituan models or lots of other models.

What does the benchmark even mean when people are using AI to make real world things that solve real world problems?

I see people, and my self making amazing things with AI and fixing old projects and having real world impact at the fraction of the cost it would take me to hire people, or hours spent on my own coding.

I have built tools and systems with AI that have allowed me to build windows drivers, android apps, web apps, iOS apps, vm occultation, custom block drivers, custom file systems and more. To the point where entire products have been created.

Not trying to be a doomsday, but yes. It seems as though with the right infrastructure we are at the point where businesses owners can go from idea to product very fast and not need or hire much external talent.

> Im sorry to hear about your situation. Have you blogged about it?

I don't know why but this made me laugh out loud. I know you're trying to help but just a funny jump

I stopped blogging years ago when I realized that only bots were reading the posts.

I think the volume for custom agents is probably higher than ever but so is the competition for that work.

That working is as not common as college dropouts becoming successful entrepreneurs. But both become stereotypes. But I know you might also just want to read about it.

PS. Not everyone (in fact the rare few) write as if no one is reading. For most, literally no one ever reads :D

bingo. to give an analogy in terms of law firms - you can either be a partner who brings in new business or the associate who does the grunt work of reviewing/writing contracts.

guess one which is valuable ?

The problem is Claude Fable is now better than most programmers I know at software architecture and performance optimization as well.

That's a popular type of AI cope.

First, I am aware they have been able to write code for some time. I made my first LLM coding agent experiment the day after ChatGPT first blew up in November 2022.

I haven't been a "code monkey" in the last say 15 or 20 years of my programming. I would argue also that code money was never really a thing, it was just an excuse that pretentious overpaid developers used for the existence of underpaid exploited programmers. 95% of programmers have to do requirements analysis and design regardless of their pay grade. I learned a long time ago how important requirements analysis and good design is. And the jobs I am applying to largely specify architecture and requirements analysis in the description. Employers are aware of code generation tools.

But also, the premise that writing the code is not the hard part is just nonsense. Sure requirements analysis is harder than people realize, good design is hard and so is understanding things about proper and closed iteration. But that stuff is not, in the aggregate, harder than the code.

And also, frontier models can absolutely do requirements analysis and architecture. And the sheer speed of implementation means that there is a huge built-in advantage for iterating more effectively.

Can you give me a hint about wtf you are talking about?

Not sure what you are trying to say but thanks for reminding me about qm, it has some cool features.

My MindRoot framework had some of qm's main features awhile ago though. I do try to use it to build solutions by customizing rather than from scratch when possible. But a lot of clients or potential clients don't like that idea even though it's MIT. They want some that has clearly been invented by them as a new programming project.

I actually turned down a project last year from someone who wanted me to start over in LangGraph -- he had already decided on it before he knew about my thing.

The most recent project I have been using MindRoot and building up the voice capabilities so I can now handle voice agents end to end including SIP.

But on that website now I try not to emphasize my framework at all for most proposals. You don't have leverage on Upwork. And also with code generation it isn't a critical point necessarily.

Qwen-3.6-35B-A3B was our "gateway drug" into switching our organisation to agent/harness-first coding.

Particularly, I had one team member who was extremely sceptical of AIs/LLMs/harnesses and refused to use them. One day he said "Well, I have an RTX 5090 doing nothing... should I try to get something up on it?" and a few minutes later he had 3.6-35B loaded up, running OpenCode.

It continues to be a workhorse to this day, running on both my local Mac for various types of jobs, an AMD R9700 at the office, and said teammember still uses it on his 5090, although in practical terms we do a lot more with DS-V4-Flash-0731 these days.

35B MoE is certainly a good and fast local model. I find 27B dense to be quite a bit smarter, so I daily drive that. I wish there was a ~100B MoE with maybe 10B active. It would be super smart and fast!

It was for me too but the new deepseek pricing is too good to ignore for now.

I honestly think that with my electricity prices running qwen 36B myself is more expensive than hitting the cache rate at deepseek.

I would recommend looking into Ornith1.0 - it's using Qwen3.6 35B-A3B and excels in coding, at least for my coding needs, Python, web-dev, SQL scripting and some C#. Using Pi harness.

I have been using Qwen3.6-35B-A3B as my daily driver as well and its been phenomenal when it comes to coding

What kind of machine do you have running that? My attempts at local have always resulted in a very hot lap

With laptop being ...?

Is your Qwen3.6 locally run on your laptop? What kind of tokens/s are you getting from your laptop GPU?

compared to claude - how 'fast' is it in terms of throughput on your laptop?

What do you use to pair it with web search?

I'm a big ole noob when it comes to local AI. What are you using for a harness? Or platform to interact with it?

On what hardware do you run the model locally, if so?

What are the specs of your laptop and what tokens per second do you get?

I'm on the verge over here, the new Anthropic models have been a disappointment. I've tried the A3B variant, but had mixed results. What do you use as the coding agent, and have you heavily customized your workflows?

What laptop?

Kimi K3 is more like "weights available" in that you can download and use them but it is under a custom license that has a bunch of limitations where you have to pay Moonshot for doing some stuff. GLM 5.2 on the other hand is plain old MIT.

Not sure how Qwen3.8-Max is going to be licensed, hopefully it'll be Apache like the smaller ones.

US AI companies are already sweating and 100% pressuring the Trump administration for more anti-Chinese regulation, since there have already been talk of Trump considering banning Chinese models. There's however another push back from the startup industry urging them not to ban it, since it will stifle the innovation. In other recent news OpenAI also greatly cut their model prices, 20% for 5.6 Terra and 80% for 5.6 Luna, to stay competitive.

And if you're running it on a dGPU, power limit it, because you lose very little in terms of token generation performance, since it's memory-bound.

Oh. I've been using an icepack under my laptop to keep mine cool. I'm watching it with llamatop to see if the GPU is actually active or not, aw activity monitor wasn't showing me what I wanted.

How and where do you finetune it?

I also "evolved" into 27b (q8 unsloth) and pi.dev (tried many combinations) feels for me the same as opus 4.5 that i use at work, faster even (using 2x 3080 20GB gives me 60-80tk/s). Though you do need to feed it more details up front (about what exactly you are planning to do and a good written skill.md) but I work that way anyways, im hyped for 3.8

All of us with a 96gb rtx 6000 would love to see a 70b moe. Maybe they are waiting for OpenAI and Anthropic to IPO so they can short their stock and release. Local LLM is going to get very interesting in the next 2 years.

> what level is this compared to other Western paid version?

IMHO this is a difficult question to answer. Part of the power of paid models comes from the software supporting it. With local models, you have tons of workflows that can severely influence the quality of the result.

In my personal experience, the SOTA models are way more consistent and can handle more complex questions. Part of that is (probably) because I don't let my local model access the internet, while paid models do use the internet to look at docs etc.

Yes let’s get rid of employees so no one is employed but somehow they can afford to buy my stuff

The larger 3.6 35B is actually a mixture of experts (MOE). This means a small proportion of those B's are actually active. It's fast and suitable for agentic tasks but nowhere near good as the dense 27B model, which has all of its parameters loaded.

I have found 27 to just be so much more coherent than 35:

https://humanparadox.org/local-vs-frontier-benchmarks-for-my...

It can complete multi-step tasks much better, and has a bit more curiosity.

https://xcancel.com/Alibaba_Qwen/status/2084100707423289643#...

Which is why big labs have been working hard on making their harness not be stateless any longer:

https://earendil.com/posts/session-portability/

“Just take the session thread to another provider” might not be feasible anymore soon-ish.

They can be deterministic. We did this at Groq, if you sent a request with exactly the same input token, seed and temperature value you would get precisely the same result every time.

This is harder to do on other architectures that themselves aren't fully deterministic though.

I appreciate the replies on the determinism point and I’ve learned some new things here. In any case I probably should not have tagged that on, as my main point was to share that the sort of property that parent is talking about (whether true for all LLMs/providers/harnesses or not) is statelessness, not idempotency.

LLMs are, in theory, deterministic. Sampling is not intrinsic to LLMs.

Greedy decoding a single batch in most libraries will give you mostly deterministic outputs. Higher batch sizes can increase variance.

But all of this is down to CUDA and/or kernel implementation issues.

According to [1] there is - unfortunately - increasingly more state in LLM sessions.

1: https://earendil.com/posts/session-portability

>LLM responses are certainly not idempotent, as they are not even deterministic.

Isn't that more due to an optimization and not how the LLM itself runs?

Like a MoE LLM run on a single input should give the same output each time. But this is inefficient, as any given token is hitting 1 (or maybe 2 or 3) experts at a time, meaning all the other experts are doing absolutely nothing. So you upgrade it to take in multiple requests. But then any given expert can become a bottleneck, so when too many requests need a given expert, some of them are routed to a second or third best expert instead. Within the context of any single request, this looks like non-determinism, but it is still deterministic when considering the full batch.

For everyday users and everyday use cases, that is enough to treat it as non-deterministic (the harness might also send in unique data like current time which means one can never have the exact same request twice), but when talking about LLMs more theoretically, I think we need to consider they can still be ran deterministically even if that isn't as optimized.

Similar with temperature. 0 means deterministic, but anything higher with a seeded value is deterministic. If anything, temperature is us purposefully adding non-determinism to agents because they were too deterministic.

Moat is not the harness. Harness itself is temporary until the models get better and slowly the code in harness will go down.

Note that the biggest GPU providers in the world are the hyper scalers and even they couldn’t allocate more if you pay for it. Because the rich companies and well funded ones are gobbling them up to the point where if tomorrow a 5T model that smokes every other model in the world is released you just can’t afford inference.

Surely the moat is the training data... with the data you can explore new architectures much easier and get step changes in performance.

Is it RLHF if there is no human feedback?

And yet investment is continuing. What are they counting on?

compute is not a moat, it's a rapidly depreciating physical asset. buying up all the shovels in a gold rush does not give you a moat, it gives you a slight advantage for the time being. someone else will just start making shovels. and the data is clearly available, hence the number of open-weight models.

your post helped me realize a change Meta is pursuing on Instagram that is to give more weight to captions and long text posts so they can have more data to training that would usually go to websites/Google. Even AI slop is good for this.

> They need to start moving parts of the harness behind the API

This isn't without it's challenges however.

1. This will increase costs drastically, since they would need to a run a sandbox per use to ensure data isolation.

2. Increased latency, and this directly limits how much of the harness can be moved to the cloud before the users notice sluggishness

There's no mandate that says any of those bets are going to generate a return

That might be true right now, but how long until you have to move the goalposts? In my experience with DeepSeek and Kimi, they're as capable as the frontier was four months ago, which already solves a big chunk of the coding tasks that I'm interested in.

What provider did you use ? Synthetic's Kimi is a beast

Also tried Qwen Code. Tried it with a coding plan and it 401'd. Tried it with an api key I loaded up with pay-as-you-go credits and it 401'd.

It did a self update and it worked for a single request (me just saying hi). Pasted in the prompt to build the page and it 401'd.

I thought it was pretty cool, its the kind of stuff that I wish there was more of on the internet. I guess some people's imagination runs a little more than others.

Yeah, I'd assume it's possible to extract all languages as steering vectors from a model and then substract the ones you don't need from its weights.

However, that would just change the weights values and not their dimensions.

Can you please explain why?

Actually, it's shown that even general knowledge helps coding models because their input is natural language itself so they need to understand it well enough to even turn into code.

I think this is really a rosy image of China’s electrify situation. They have huge needs in the east and green energy in the west, they can’t build their UHV links fast enough. They are doing ok at keeping up, but there isn’t a surplus of cheap electricity lying around in the same places where you have cheap water as well.

There is economic benefits of the cost per a lumen continuously coming down.

The US also isn't energy-limited. They'll just price out all humans living there from buying any electricity.

EDIT: This was obviously meant to be facetious with the intent to highlight the negative effects on the general population that datacenter buildout has in the US.

> but the need of the many come first.

The need of the rich, you mean. In these kinds of situations it's never about the many, if it was about them different choices would be made.

> lack of European competitors

There's Mistral. Although they've been surprisingly quiet for the last 6 months.

Theres a pretty big chance imho that europe will follow suit , or at least the segmenets that are fully aligned with the US on all things china even to their own detriment. Remeber rip and replace Huawei 5G because of national security , leading to some parts of europe having worse coverage than some tiny african markets. Best bet would be for europe to look at it like what it is an open source commodity that is useful to build on rather than a moat to extract rents with.

well, the EU just brought down the regulatory hammer which complicates things a bit. we will have to see if it operates as a proxy ban on open models depending on how they go about enforcing all the safety, bias, transparency and data provenance requirements.

I mean there is Mistral?

>Europe due to its lack of European competitors, has no current similar protectionist incentives [...] The genie is out of the bottle in the West already.

I agree, but you are underestimating EU regulatory incompetence

Send you to jail.

What they always do. Send in armed men with guns? Export Controls. Import Controls. National Security Laws.

Level 1: Anyone using unauthorized model weights in their business is prohibited from doing business with the government...

Level 2: ... or with anyone who does.

Level 3: Anyone caught with unauthorized model weights is subject to the same prosecution they could expect if they had been caught with CSAM.

In Qwen's case, both 3.8 Max releases seem rushed may be in response to Kimi K3 getting all that attention.

3.8 Max Preview ran 90% discount on QwenCloud exactly as when Moonshot couldn't keep up with all the signups.

It gave the svg for the wheels in the reasoning trace then forgot to include them in its final answer. Lol.

Wow, bike geometry is really good! Except for the missing wheels lol

K3 also forgot the wheels when I tried it, but it turned out to be a bug in llama-server's built-in SVG viewer. I know you have your own rasterizer but just FYI, sometimes it isn't the model's fault.

Seems to me OpenAI and Anthropic are kinda following the Apple business strategy. Those two offer a premium service that gets better results and works more seamlessly. I.e. the integration between Anthropic models and Claude Code is apparently nice and gets better results, and I've heard anecdotally that Codex is currently the best.

So just like in IPhone vs Android, you could end up with a situation where Chinese firms compete and get most of the revenue and usage with low profit margins but OpenAI and Anthropic capture a premium side of the market and still get a lot of profits.

> In the interest of perspective, can anyone (perhaps playing devil's advocate) give one?

I have numerous cases where Sol failed and only Fable could solve a problem. For example yesterday I was merging a Q2 curved with a Bezier curved face in 3D using OpenSCAD. I tried for over 2 hours with Sol 5.6 high and x-high.

Fable two-shotted it in about 30 minutes.

In my experience open models (or GLM, DS and Kimi) are radically worse than either of Claude or ChatGPT at these tasks.

I think there is a huge "long tail" of tasks like this where the frontier labs are ahead, and I think this long tail is quite important.

Hardware and electrical costs including power usage and electric wiring/outlet costs of such machine.

Unless you are spending more than a max subscription (200 a month+) its cheaper to use the cloud.

But things are priced cheaper in the cloud now to lock you in and restrictions around hosted models are getting worse.

If you only have a $300 dollar laptop its probably not worth the upgrade.

I'm personally excited by local AI but the experience for the average isn't the same. I'm willing to get .5/s running on 10-15 years old machines but what I can do with it is limited.

> What am I missing?

Their marketing department :-) . I'm only half-joking; those guys are hard at work finding the best product-market fit for ChatGPT/Claude. "Product market fit" means "strongest revenue", which is not necessarily going to bring the best tool for you or me, but the one that can either get more consumers to shell off money, or more enterprises to cough money for licenses, and in both cases those consumer basis will be narrowed down to what legal and geopolitical circumstances allow OpenAI and Anthropic (and this is why they want to ban the competition!). It also means dark patterns and enshitification, of which I'm already seeing some both in the Codex interface (it was just renamed "ChatGPT"!!!) and in Claude Code (which also is just "Claude" now and can't '@' properly any longer). So in the medium run most people will be better off running an open source harness that can use any model.

coding on a laptop is only one use case

you can't create a new drug by running a model on a laptop. You can't serve a customer support bot running on a laptop. You can't generate video in bulk for many users on a laptop. So there is still a case for paying others for inference.

Does it justify the valuations? No idea, but some major use cases are still there. That's why they are rushing to implement, OpenAI creating a "deployment company", Anthropic having some pharma rumors, etc.

Fable 5.1 is coming, then GPT-6...

You are right to feel that way about the frontier labs, especially Anthropic. From https://stratechery.com/2026/anthropics-safety-superpower/

> "Anthropic believes that they are the ones who should have final say over how Anthropic is used; given that they think only they should be developing leading edge AI, they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."

If there are genuine society risks in a tech I don't want to discourage CEOs from talking about them. I feel like we've spent decades talking about how evil chemical companies (etc.) were about covering up issues in the 20th century. But yes, that's different to being a reason to ban external models.

Sam drank the "superintelligence" kool aid early on and said 30-40% of jobs could be impacted by AI, but recently admitted he was wrong

> “My scorecard, at the highest level, would be we’ve been roughly right on technological predictions and pretty wrong on the social and economic implications” https://www.cxtoday.com/ai-automation-in-cx/sam-altman-softe...

I agree re: Dario quietly pushing for government control. He also said LLMs would replace a lot of entry-level information jobs, doubling the unemployment rate from 4-5% to 10%.

Yale did a study recently showing little impact on employment in high-AI exposed jobs https://budgetlab.yale.edu/research/ai-probably-not-yet-reas...

But the US are the country of freedom!

That's what Hollywood has been telling me my entire life!

If k3 is open-weight (and it's available on hugging face [1]), how could they force providers to sing an extra (secret) licensing agreement?

Edit to answer my own question:

License file [2] states:

> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.

[1] https://huggingface.co/moonshotai/Kimi-K3

[2] https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE

Thanks, totally missed that!

https://en.wikipedia.org/wiki/Jevons_paradox

OpenAI can't even build Codex app (not cli) for Linux... I thought code was solved?!

Ironically it's Electron BS, so actually, Sol could probably build itself...

The models are good even by skeptics standard, it's just that evangelists are overselling the capabilities. If you understand the limits of LLMs not using them as a business is shooting yourself in the foot.

However, they are not at the point where they can effectively train themselves, nor did they are capable of researching their own method of learning. SWEs in mid-corps on my country are right now relegated to reviews and sanity check, basically babysitting the LLMs and making sure they're not spouting nonsense. If you think about it, that's basically QA and can also be delegated to another AI. If Bun's rust rewrite that they tout as fully LLM-led can pass the test of time in a year or so I think that's it.

I believe all that is now constrained by compute and capital, not tech.

The world never runs out of problem. There is so much work to do.

I mean its pretty obvious right? This models are not flawless and sometimes reach stupid conclusions so there needs to be some one who watches it. Thought i must say u are right. Every one of them pretends that this new model is gonna finally take ur jobs lol

There's infinite work to be done, so higher productivity makes people worth more. (Obviously this doesn't apply if AI can do everything but we're not there yet.)

Source?

Being 16, 17 and 18 and not working is expected, given they are still at school. A good percentage of people aged 18 to 24 are studying full time.

This stat seems a bit misleading and doesn't support the "you can live and not work argument". How many of those people are retired? How many of those are under 18? How many of those simply cannot work? How many of those live at home or are on food stamp?

But how is that possible? How can one survive without income barring homelessness, begging, prostitution, and other not so utopian options?

In the 1800s most families only had a single provider, so this percentage may have been higher back then than it is now.

Maybe you should ask AI to review your statistics and reasoning. A large amount of 16-22 year old kids are in school. And retirees 80+ are retired. I don't see how any of this is relevant to the idea of what would happen to the economy if there is an employment collapse. I don't know that this would also seem like a utopian dream to someone in the 1800s; I'm not sure what that means. I think someone from the 1800s would be surprised by a great many things and that statistic would probably be low on their list.

No I haven't had time to spend my afternoon rock climbing while ai generates documentation.

It's infinite work, I just did more work while codex was doing it's thing in the background.

We are in the golden period where this IS possible. Once it becomes the norm to "do something else while your agent works", we'll be asked to do more WORK while the agent works, rather than do hobbies/housework/nap/etc.

So you were working 10 x 1 hour = 10 hours before.

And then you were doing 100 x 10 minutes = 100 / 6 hours = ~16 hours & 40 minutes.

in general with automated code reviews I've found it fruitful to tell the model what specifically to look for and where. I usually don't get over 10% of the claude opus context window for code reviews, but it of course depends on how your code is structured, how much the agent has to explore etc.

If you're literally using it as code autocomplete that's not functionally true. But it's still neither well specified nor a good idea.

Domain modelling as well: we have clear programming concepts, but the kind of autocomplete that can autocomplete a customer sales process needs to be able to refer/leverage ontological connections on words like ‘bass’, ‘season’, ‘rebate’, and ‘jamboree’.

Intuitively it feels like focused models should be better models, but human programmers are ‘better’ knowing programming alongside general stuff. We’re not reduced by reading economics or Tolkien, removing such knowledge would be premature optimization.

we do though and it can be stripped

Linking here since the pruning method you're talking about wasn't the top search result when I looked: https://github.com/CerebrasResearch/reap

Bistromathics

https://www.hhgproject.org/entries/bistromathics.html

Points are starting to be made in favor of value, to the contrary of what you are affirming. Specifically because the new open weights models lower the TCO of hardware in an environment where new open weights were previously thought to be a thing of the past.

> The point of local models is privacy, offline use, and maybe no guard rails.

But also, control and consistency. A local model cannot be changed out under your feet like an API model can be.

There is for $/creativity/token. LLM sampling settings are poorly supported even in open source serverless providers but are the single best lever you have for getting better outputs in regards to creativity (and quality for long context or highly quantized models).

We all ASSUME that.

For all we know, inference might be dirt cheap, they might just be hiking the API prices so high for us to think subscriptions are subsidized.

Yes, I don't. I have my own harness where the agent is only able to do a set of predefined things and none of them involves the internet. However, I imagine the model could be a lot smaller if it does not have the baggage of coding and programming in general.

I mostly just need it to mostly be a very advanced NLP model that is able to figure the set of tools that it needs to call with what argument. Then it will just follow a predefined decision tree.

I'm oh so seriously tempted to switch at least for a month or two.

I mean sure but I’m curious to what extent and why it’s so obvious to some. I haven’t seen this sort of language in a marketing post before.

Page won’t load for me it’s just grey bars fading back and forth forever.