GPT-6 Astra in code review: Gains, privacy, and cost (coderabbit.ai)

48 points by cebert 7 hours ago

eyalitki 3 hours ago

Comparison was done in the scope of coderabbit AI code review tool, which sadly makes it practically irrelevant.

My personal experience as a software engineer, and a former security researcher who did manual code audit, is that this code review tool has such poor results that it isn't worth the "noise" and friction it causes developers during C/I code review

maxdo 2 hours ago

They do catch important things but it’s really contextual. You can’t grab a model slap it on top and say code review . Hence a dedicated review tool is almost dead . Code review should be part of your pipeline and consume test results from the original task , open spec etc . If you do not have that code review will not help if you do , what is the point of task rabbit just slap <your harness in the sandbox> review against <goal>

stingraycharles 3 hours ago

Yeah I personally don’t understand the point of AI code review tools all that much, as AI is already generating the code as well. All of these AI code review tools create so much noise, yet don’t catch the really important things.

pinkgolem 2 hours ago

What really important things are human reviews catching in your org?

I just feel more and more like the effort invested in manual reviews is not worth it

grokys 2 hours ago

jiggawatts 2 hours ago

Code review tools are designed for less organised dev teams that don’t do PRs and mandatory human reviews already.

It is papering over a lower level of competency without having to invest in actual human oversight or real process improvement.

9dev 2 hours ago

bitlad an hour ago

It does add lot of noise after a point you start ignoring the suggestions and findings.

Code generated these days with fable and sol are near perfect. What issues they might have is logical errors.

OtomotO an hour ago

> Code generated these days with fable and sol are near perfect.

If you're doing a simple CRUD app, sure.

If you're doing anything more involved they get the job done with dozens of shortcuts that bite you in the ass the moment you have on-call duty.

Way too much code and repetition and hacks.

Especially in GPU code, but also in other fields.

ramon156 4 hours ago

Both OAI and Anthropic seem to have released a model that is slightly better but cost ~2x the previous iteration. Interesting play

villish 2 hours ago

That likely won’t change if other competitors don’t take the lead at some point. If companies are willing to pay top dollar for the best models AND they get to extract as much money from Chinese labs distilling Astra/Fable it makes no sense to lower prices. Obviously not great for everyday users who don’t have unlimited money.

kzrdude 3 hours ago

That should be expected based on the scaling laws that we expect; larger models are more intelligent and cost more. Now it's very unfortunately that they don't publish the size of their models.

arthurcolle 3 hours ago

Astra and Sol are the same price when you factor in token efficiency

Squarex 2 hours ago

I don't know, in the Codex app, it burns the limit much faster.

sscaryterry 2 hours ago

Bullshit.

jstummbillig 3 hours ago

Roughly how we price (high skilled) human labor.

simianwords 4 hours ago

Interesting comment because it is true that Astra is costlier for the same intelligence tasks as Sol.

But this is not the same for Fable at all.

SneakyZero 3 hours ago

Astra seems to be really slow. Maybe it intends to read more context. But from my experience it is definitely slower than 5.6 sol when handling same tasks.

trvz an hour ago

It's a bigger model, of course it's slower.

dude250711 19 minutes ago

Given that Fable is a Sol-class model, should Astra not be compared to Mythos in those tests?

sscaryterry 2 hours ago

From what I can tell, its shit.