Hacker News

> Leetcode was stupid before LLMs and now it’s even less useful as a signal.

You got my vote right there. I also find it funny that LLMs can easily do leetcode problems, because there were so many examples and easy test cases.

> You do not get a better signal about that from asking harder leetcode problems.

I think this is a thing that needs bigger formatting for this statement or a feature to vote twice.

---

Only response I have to this:

There are many different schools of thought on how to architect systems. In many cases these different opinions don't have a lot of solid backing on why you'd do this. On top of that there is level restrictions that prevent talented individuals from being more influential. I.e. an Architect is less about technical experience and excellence and more about the abiltiy to communciate to high level stakeholds and having systems that can solve the problem [some what].

To get into Marine boot camp, there's a minimum strength test.

To get into college, there's the SAT test with a minimum score.

To get a high paying software job, there's the leetcode test.

These are gates meant to filter out the obviously unqualified candidates. Unqualified candidates waste the time of the candidate and the company. The real interview comes after that.

You missed the point. I’m not saying I support dsa but you are arguing against the weakest version of the reasoning behind why companies use dsa. It’s used as an iq test. IQ is highly correlated with many things associated with intelligence yet even though we don’t take iq tests on a daily basis it still correlates.

The argument was dsa functions like an iq test. It’s not relevant to the job (in fact it’s harder than the job, far harder) such that if you do well on it companies can be confident you can do well on the job just like how scoring high on an iq test makes people confident you’re intelligent.

There are plenty of arguments against what I said above and going after this argument is the better target because the irrelevancy of dsa to the job is widely known and obvious.

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

This doesn't really say anything about someone's familiarity with git, considering pull requests are not a feature of git.

When I worked at a FAANG we used git but had our own internal code forge that was nothing like GitHub.

I feel like git operations are one of the few things that I truly don't care at all about whether someone uses an LLM for and doesn't tell me. PR descriptions and commit messages should be written by humans for humans, but like, if someone doesn't know how to manually configure their git remotes and what exact command to use to amend/squash their commits into the shape needed and instead has an agent prepare all that and use the GitHub CLI to open the PR, why should I care? Beyond the code itself, I care about the things that will continue existing in the long run fitting my team's conventions (the number of commits, the commit messages, the PR description and comments, and then whether there's a merge commit at the end), but the process used to create those things doesn't matter to me.

  we also ask the candidate to open a PR against the repo just to see if they know the basics of git 

A candidate may be able to open a PR and not be able to make a commit without an IDE.

Another candidate might know plenty of git, but haven't worked with Github.

I run a live coding challenge on a rather benign coding problem with the explicit expectation that candidates will use an LLM.

Interestingly, an LLM involved feels like it’s made it easier to assess candidates. Senior candidates know exactly what they’re doing and strongly driving the LLM. They pre-communicating design decisions, gotchas, and requirements. They also ask a lot of clarifying questions and dial in the scope (after all not building something is better than building it).

Less experienced candidates tend to jump right in, build way too much, and stumble because they delegated all of the critical thinking to the agent. They often end up in a place where the agent doesn’t align with their vision and they can’t really reign it back in.

Git is a simple tool. I expect any engineer can learn to use it. While git is common, I would not expect an engineer who worked at a company using a different version control system to know how to use git (which is a really awful - I still miss mercurial). I expect you to know the general concept of version control, but the specific on is a tool I expect you can learn quickly.

> we also ask the candidate to open a PR against the repo just to see if they know the basics of git

I've heard this multiple times now and it still boggles my mind that git is something you have to test for.

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

How does that work? Git has no concept of pull requests.

What I though of was to give candidates a not-so-smart LLM, one that is almost guaranteed to make mistakes.

The idea is to see how the candidates deals with it. Maybe he will just say "fuck it" and do it by hand, or patiently point out every mistake so that the LLM gets it right. Both indicate competence, what you don't want is someone who doesn't see the mistakes and hand you slop.

How would you prepare for the coding problem as a senior? What are the assumptions that underpin this?

> (we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it).

Worked with a "super senior", lead architect. Management was raving about the guy before he joined. Insane resume. Skipped the regular interview process because of this.

Had to show him how to open a PR and setup his git config so his commits are linked to his account. And then his PR's failed the automated ci checks for descriptions and titles, and he came to me to ask how to fix it. The failing gh action literally says what was broken.

This is all with Claude available btw, so one could literally just ask.

I guess it's a bit of a modern fizzbuzz. Where it's so simple but still manages to stump a lot of candidates.

    > we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it

Funny that about 80% of the replies to you comment are about your git sidebar. Bikeshedding indeed

how do they grade a right answer, suggested by the agent itself??

Something similar for senior devs: code reviews. Reading and understanding code has always been important. I find offering simple code reviews very telling. In a way, your LLM approach matches that. It's a way to do a code review. It's a test of understanding, and not just memorization.

> we also ask the candidate to open a PR against the repo just to see if they know the basics of git -- you would be surprise how many people cannot do it

Will you allow jj instead. I switched to jj 1.5 years ago (as many others have), and my git is now rusty.

I mean sure, I can do add, commit, basic diff, etc. But rebase/merge - oh dear, I forget which one is source/target (jj made the infinite design choice that the ordering doesn't matter - you have two parents making a child - it's symmetric).

>>> the bar is super raised and now you need to explain in details everything the LLM wrote as if you wrote it yourself.

You have missed the point entirely.

> Almost none of them have shown even the slightly indication that they are aware there there is scientific research on what works for an interview,

The topic of the article is about adapting interviews in the age of LLMs.

Scientific research on interview techniques requires years of retrospective analysis after the hired employees have been evaluated in their roles for a long time.

That research does not exist on this topic. It's too new. I don't think this is a valid criticism of the actual article.

They do say "Not every onsite includes these new rounds yet, and when you come on-site we’ll give you a heads-up about what to expect."

So that implies that they are testing this, trying to determine if it works better than the current process. Whether they are doing this in a scientifically rigorous way is unclear, however.

I never see this question being asked enough.

In my experience (both as interviewer and interviewee), interviews primarily serve to satisfy the ego of the interviewer.

I used to interview people and would generally just have a chat with them about previous problems, projects, or experience at school. I would work out a pseudocode problem with them (designing an elevator). From what I can tell everyone I said yes to has had a fruitful career.

The idea of forcing a candidate to give a third party access to their screen, transcribing AI notes, and forcing them to do 3+ leetcode, and an architecture round, and a behavioral round while I watch over them for hours on end sounds completely idiotic and antisocial. This entire industry is high on its own supply when it comes to interviews.

Even Steve Yegge got selected (or maybe nearly selected) out by his own process (unwittingly, mind you)

> One day, the recruiters gave us a special round of packets to review. In these special packets, we were able to read the interviewer notes and candidate responses. All personal details were stripped out, and we were told it was a “calibration exercise.” We had to do our regular voting job with these special packets, and see how it went. I think we may have assumed they were from another site, since cross-site calibration was common.

> Our group did our job, and voted not to hire about 2/3 of the packets. This was about par for the course.

> But surprise surprise, this time, those were our own packets from when we had all interviewed at Google. The recruiters had tricked us into reviewing our own interview packets, and we had voted not to hire most of our own group.

From https://yegge.ai/essays/the-last-technical-interview/ (May 2026)

According to algolia, there were 57 posts with jane street somewhere in the title or url in the last year, so a bit more than one per week. For a Poisson distribution with rate = 57/52, P(X >= 4) ~= 2.5%. So there should on average be ~5 weeks with 4 or more jane street related posts at that rate.

Probably just random fluctuation.

Probably a self fulfilling prophecy.

Jane street news made it to the front page a couple times and got a lot of attention.

Now more people are recognizing the name and upvoting other Jane Street posts.

They’re pretty techy, idk why they’re showing up recently though.

Also one of my absolute favorite programming talks ever was hosted by Jane street:

https://www.janestreet.com/tech-talks/algorithm-for-precisio...

It’s like half talk, half story. Got me super interested in logic programming, and minikanren in particular.

I'd lean towards no on that. Might be a bit of a Baader-Meinhof thing. They've been mentioned/discussed on HN pretty regularly for years:

https://hn.algolia.com/?q=jane+street

People like to talk about Jane Street because it is this Very High IQ™ company that pays people astronomical salaries and is extremely hard to get into.

They don't have to do marketing, though. It's not like they sell anything to Hacker News readers, they make their money trading finantial assets

> We know this sounds vague,

ok thanks for this slop then. Dont see why this stupid article is on frontpage.

They seem to belive in 'organic' marketing like doing a fake podcasts on dwarkesh. So maybe they have some bots upvoting their name.

I posted this. I'm just a regular dev, without any kind of attachment to Jane Street, in fact i'm a little wary of them, and would subscribe to Evan Czaplicki's understanding of the faustian bargain working there would involve. I think they're banned from India, if you want an example of the kind of raiding they are equipped to do.

In general - I am wondering how engineering interviews change in the age of LLMs, Jane Street, tech-wise is a "thought leader", at least having opinions, sharing them is part of how they recruit "the best", i think oxide does this too.

So in some way, in this time of change it makes sense that they are noisy... It also makes sense that the noises they make may filter down...

There are three Jane Street stories on the front page right now. This one, "Socratic Tutor (github.com/janestreet)", and "A false smoke alarm almost drowned Jane Street's only NYC datacenter (bacarella.com)".

What I learned over time is that first, the pool of people who regularly submit to HN is surprisingly small, so if they have interest in a particular set of topics, these topics are overrepresented in the queue; and second, relatively few people patrol new submissions and the the first 3-4 upvotes in the first 15 minutes decide if something bubbles up to the front page or not. These folks often go by keywords / vibes / employer associations, so again, certain topics get an advantage that's unrelated to the merit of the story.

More generally, I think there's been a significant drop in topic diversity on HN over the past year. Almost everything on the front page is related in some way to AI or to companies on the forefront of AI. It's almost as if there's nothing else going on in tech or in the broader world. If you go to some other aggregators, you get a different view.

They pay a LOT of money, have some high level of “nerd” prestige for solving interesting technical problems, have good outreach with things like problem of the month, and have not been impacted by massive layoffs.

There are pretty few companies out there like this. Being relatively small probably helps; everyone knows someone that worked at Mag7 and maybe has a bad story to share, but this is more rare.

All three stories fit the criteria of gratifying one's intellectual curiosity, which is goal of Hacker News submissions.

Consider this: Would you find anything unusual about seeing those stories on the front page if each came from a different company or didn't mention the company name at all? I wouldn't, because each of the submissions is primarily interesting for the content, not the company associated with it.

I think Jane Street has gained prestige and notoriety in the last few years. For people like Math majors who are split between tech and finance, it’s the most famous I think.

Noticing a bit too much

What is the deal with it though? Daily front page content in various forms starts to raise eyebrows.

That may or may not be suspicious, but Jane Street has been posted here a lot since forever, mostly because of ocaml.

> One company give access to LLM (probably custom) that would provide wrong code intentionally and candidate has to correct it.

How ‘wrong’ are we talking? Like the code is correct, but slow? Fails easy-to-spot edge cases? Uses poor taste in variables names?

This is probably genius and also hilarious!

that last strategy is amazing

> One company give access to LLM (probably custom) that would provide wrong code intentionally and candidate has to correct it.

"You are a helpful coding assistant. Make some mistakes."

I had lunch with a couple friends this week, and we each work in different tech environments. Most of the conversation was about the workflows we're each using, and seeing our colleagues use.

It's a really interesting time to work in this field. People who are paying attention to workflows are using a range of tools, and constantly refining their own workflow. For example a whole team might use Claude, but one person collects data about their own usage on an ongoing basis and develops more advanced workflows than everyone else on the team. Their focus isn't just about working faster, but about doing better work.

I don't think the post is vague; I think we're at a time where you need flexibility in your interview process to find candidates who are doing the best work these days.

They're basically doing what everyone else has been doing for years (minus the LLM part). Just focusing on software and system design rather than coding riddles.

Don't worry; their process will still be absolutely brutal.

Well what can they bring to the table?

The alternative is to abolish interview altogether. No more hiring pesky annoying humans.

Upvoted this!

Their literal tagline is "Work where your mind matters". They advertise to mathematicians, competitive programmers, FPGA engineers, systems programmers, all in search of a non-bullshit job where someone can appreciate their true skills and somehow advance their field's frontiers.

The LLM era is just dissolving this illusion, revealing the more benign truth: they are a financial company with mildly unusual tech needs. Just like Y-Combinator is just a venture capital fund, just like big techs are just a dysfunctional companies... Your work there wouldn't change the world, and if we left our future up to these institutions, we'd spend it in a couple decades of glut, leaving a deindustrialized barren husk for the next generation.

This is clearly an advertisement, so I'd say they're successfully getting their marketing onto HackerNews.

Money.