Reverse engineering my ADHD test (nullpt.rs)

164 points by hazebooth 4 days ago

usernametaken29 5 hours ago

I spent a decent amount working on ADHD diagnostics and unfortunately it sucks. To my knowledge, to this day, there isn’t a single reliable biomarker that identifies autism spectrum disorders. It’s usually correlation studies paired with questionnaires which for lack of better words can be described as “incidental” at best. Most z scored tests I know are also developed with small samples, a hundred participants with an outgroup of 20 would already be a LARGE sample. The standard ADHD questionnaire was validated on some 35 people.

There might be some hope in deep learning, particularly in observing eye movement components to deliver reliable biomarkers in the future, but I’m not so confident any time soon.

It’s dreadful and fascinating at the same time that defining and testing for ADS has been so elusive for so long…

timster6442 3 hours ago

From a molecular neuroscience perspective, for both ADHD and ASD (autism) you're unlikely to find any pure biomarkers as clean as a disease like huntingtons disease whose mechanism is clearly understood as extra CAG repeats on the HTT gene (a clean mendelian disorder). ADHD is defined by behavior/cognitive symptoms and most all behavior/cognitive diseases are multifactoral, which ADHD is. WES/WGS and GWAS has been done and for familial ADHD and unlike ASD there is mostly no clear monogenic mutation (a single gene that would explain the phenotype) [note monogenic mutations are usually a very small subset of cases but give scientists a look into what pathway/mechanism is causing a multifactoral disease]. Thus it is not for a lack of looking classically, and deficit is speculated to likely manifest as a neural circuit organization or synaptic issue (though the latter is more closely linked to ASD). This is currently impossible (basically) to measure in alive human brain as it requires nanoscopic imaging, note how challenging the fly map was (incomplete IMO as well).

genxy 3 hours ago

ADHD also evolutionary advantages, so there is unlikely to be a single gene that encodes for it.

Quarrel an hour ago

anamexis 5 hours ago

ADHD is not an autism spectrum disorder, is it?

usernametaken29 4 hours ago

That’s my bad for conflating the two. Indeed they are not the same in ICT-11 but! diagnostics is moving towards the direction of treating them the same. ADHD and ASD have extremely high comorbidity and it has been proposed many times to classify them together because the symptoms overlap (difficulties at school, aggressive behaviour or temper, lack of social skills, etc). In fact it has become so difficult to find distinctive features of ADHD both in diagnostics AND treatment (both are usually treated with cognitive behavioural therapy and in more severe cases medication) that it has been proposed numerous times to drop their distinction in the ICT, but this hasn’t happened yet. Good paper to the point:

https://pmc.ncbi.nlm.nih.gov/articles/PMC4654237/

great_wubwub 4 hours ago

Autism Disorder with High Distractability?

pstuart 2 hours ago

poopmonster 4 hours ago

No, however they are often comorbid.

izzylan 4 days ago

I took a similar ADHD test. It seemed so easy and simple that I always wondered how it was actually supposed to catch me.

Good to know that neurotypical people mostly likely test differently than I did.

burnto 27 minutes ago

What a lovely mini journey. Turns something possibly stressful into something weird and fun.

_fw 4 days ago

I found myself trying to come up with a system that used the position of my hands to “solve” the problem for me.

But then I worried if my system worked, I would be misdiagnosed.

It didn’t work. I was 98th percentile.

Thanks for sharing this!

nullpt_rs 9 hours ago

this sounds interesting! would love to hear more about this

a2dam 5 hours ago

> At the end, she told me she would review my exam and have the results ready in a few days.

Yeah man I can go ahead and let you know early if you want.

johnwalkr 3 hours ago

This comment is so funny to me, because I imagine you being distracted to make the comment before making it to the end where the psych did indeed review the new results.

Well I only know that because I skipped to the end. Then, while scrolling up I realised I missed the whole side-quest about promising to not cheat the test again, taking the test again, but then "accidentally" finding an exploit the system, and then cheating the test again, but this time cheating even more to see the results.

Then, scrolling up more I see that this whole thing started with seeing stars and then buying a telescope on amazon (probably at 3am).

And here I am commenting on HN instead of replying to an email...

jaggederest 3 hours ago

Brother the test is quick, easy, and painless, and the results can be lifechanging.

The other thing is, ADHD in my mind is really a syndrome that starts with a symptom - inattention - that can be caused by so, so many things.

I have both, because of course I do, I had a medical problem that exacerbated my ADHD to a significant, significant degree (it's amazing I was able to be a software engineer for so long without treatment). So don't just tunnel vision in... as we tend to be prone to do. Get the whole workup with an experienced clinician, it's so lifechanging it's hard to describe.

johnwalkr 3 hours ago

hungryhobbit 3 hours ago

I know this is a joke, but I'd be careful about assuming: I had a trained therapist diagnose me with ADHD ... incorrectly.

It turns out, there are other conditions out there that can appear to be be ADHD. If a trained therapist can confuse them, outside observers certainly can too, so it is possible that "very ADHD-acting" person you know might not have it.

(Or, especially for females, the reverse can be true, as not everyone with ADHD manifests with the same trademark hyperactivity.)

pstuart 2 hours ago

I understand this is your personal medical info, but could you drop any hints as to the nature of the misdiagnosis? I've learned late in life that I am neurospicy and learning from other's experiences in this domain has been quite helpful.

annzabelle 8 minutes ago

nullpt_rs 9 hours ago

hi! author here, please let me know if you have any questions :-)

jaggederest 5 hours ago

No questions, but feedback for you from an engineer in a similar situation (late diagnosis), make sure you go back and get them to check for every other thing under the sun. You really don't want to dive into the ADHD side and find out it was something else (other things can masquerade as ADHD), so a good neuropsychologist is critical. And "body medicine", too, for example adrenal disorders. It's worth being extremely thorough at this point - you're off the standard path, embrace it.

Also, congratulations, once you figure it out your life is going to change for the better in a way you can't currently understand (literally, lacking the brain regions and/or development for it, take it from me), but the process can kinda suck a lot, so stick with it, it's worth it in the end, wherever you end up.

mrbenjihao 6 hours ago

Would you mind sharing the test platform?

freehorse 3 hours ago

Not OP but I think it is moxo.ai

johnwalkr 3 hours ago

I have a question (and please take it in jest)! In between buying a telescope, seeking out an opinion on ADHD, accidentally hacking an online test, and then blogging about it, how many things on your todo list(s) did you not do?

nullpt_rs 3 hours ago

I actually wanted to make a YouTube video for this as well! I also have a few more web fingerprinting blog posts in the backlog T_T

jaggederest 2 hours ago

agentdev001 5 hours ago

Have you got some fancy scroll thing for your blog? Something in my brain is noticing that scrolling feels smoother than your average site, though maybe im crazy.

nullpt_rs 3 hours ago

I do not! Other than when clicking a section from the table of contents. Is it distracting?

BearOso 8 hours ago

Following through on the top comment, surely you realized when you decided to forgo the back half of the test that, had the program compensated for it, you were consciously leaning into the ADHD diagnosis, right?

nullpt_rs 8 hours ago

Sure! However, I think at that point I just felt trapped and overwhelmed that I no longer cared about the test or its results.

jaggederest 5 hours ago

flysand7 4 days ago

Opening up devtools during ADHD test is so ADHD, almost as if he didn't even need the psychiatrist to explain the results :D

Muromec 5 hours ago

That doesn't have to be ADHD, could be the *tism too. The tells are usually somewhat overlapping.

In general, if you are in the industry, save yourself (and your kids) some trouble and get diagnosed before it becomes a problem.

Add: I wonder why they don't just give people some ritalin and ask if they feel calmed down? Except the obvious reason of course

jaggederest 2 hours ago

So this is the cheats with "give everyone ritalin and see how they feel" - it works for most neurotypical folks too. It's just a very small increase compared to the annoyance and side effects, so it's usually not worth it. To extend the car analogy from a sibling comment, putting race fuel in a volvo.

So they can't test by response, because everyone would be like "yeah I can focus better", you have to test with the drug on, and the drug off, to see whether the "horsepower increase" is worth the fuel price. Increasingly strained metaphors are my strength.

gavinmckenzie 5 hours ago

That’s basically how the end of my diagnosis went because aspects of my results were borderline, so she suggested that I just try meds to see. But keep in mind that only one type of ADHD involves being hyper. Some of us are inattentive types where our brain feels like an old 70s pony car engine, and if you pull up to a stoplight you need to keep the motor revving or it will stall. Taking Ritalin for us is the equivalent of keeping the revs up.

jaggederest 2 hours ago

acjohnson55 an hour ago

poopmonster 4 hours ago

aidenn0 5 hours ago

> Add: I wonder why they don't just give people some ritalin and ask if they feel calmed down? Except the obvious reason of course

I have been screened a few times for ADHD and each time was found to obviously have it.

Ritalin does not make me feel calmer. I have been told by those close to me that there are externally visible improvements in my behavior. At doses to reach that effect, it also impacts my sleep quality enough that I don't take it.

Adderall causes mild symptoms of psychosis and mania before it has any theraputic effect.

Dexmethylphenidate (aka Focalin) has identical theraputic effects to ritalin, but slightly reduced side-effects.

annzabelle 26 minutes ago

fv3y 4 hours ago

scrapcode 4 hours ago

Muromec 4 hours ago

pstuart 2 hours ago

jrapdx3 3 hours ago

> ... I wonder why they don't just give people some ritalin and ask if they feel calmed down? Except the obvious reason of course

In ADHD (and most conditions) diagnosis by medication response is not acceptable. Response to drugs is highly variable and fails criteria for acceptable tests. Also drugs can produce side-effects. According to professional standards, it's unethical to expose people to nontrivial risks without good cause.

schlauerfox 9 hours ago

That was my first thought. I only wish my type was to put in the work for a blog post instead of just thinking on it and getting overwhelmed with all I need to get done.

nullpt_rs 8 hours ago

Trust, this blog post was in the works for weeks and was actually supposed to have a YouTube video accompanying it that I never finished T_T

nullpt_rs 9 hours ago

I didn't think of this until after haha

rukuu001 4 hours ago

> This performance pattern^H^H^H^H^H^H BLOG POST may indicate attentional difficulties, and taken together with other findings, an increased likelihood for the existence of ADHD.

doawoo 9 hours ago

This is basically what I did to every online based quiz in school. Big respect lol.

nullpt_rs 9 hours ago

this method works more often than not :P

kleton 3 hours ago

There's a well-known issue of "construct validity" in neuroscience.

tomrod 4 hours ago

Oh hilarious and well done Null ptrs.

And brilliant picture.

sbiru93 9 hours ago

Ahahah the end got me laughing

pixel_popping 9 hours ago

Great article!

ADHD is way more complex than just an attention issue, I mean just the amount of average daydreaming and "rumination" and the constant multi-threading of thoughts... is just next level, I believe it's not properly explained to outsiders that might just think that X guy just have a hard time going in the flow but in reality it's much more catastrophic than this and will create many more mental health issues if left unmanaged.

Most ADHD people that have jumped on some sort of strong treatment (stimulants in general) are discovering a new world in pretty much 1 hour, the first time the mind goes "quiet" is incredible. I advise everyone to read more on this website https://www.adxs.org/en - it's also friendly to scrape to be used with LLMs.

ADHD isn't a good thing to have imo, it's a serious dysfunction and it's sad to see that it's being romanticized lately in social media or plainly discredited by some people/media, and while treated ADHD can make you productive "all the time", you end up addicted and pretty much permanently require drugs to function well.

LocalPCGuy 9 hours ago

I agree that ADHD is a serious condition for many people, and that it's overly romanticized and that isn't good when it comes to getting help for those that do need it.

To that point, I'm going push back on this language just a bit:

> you end up addicted and pretty much permanently require drugs to function well

That is negatively framing medication, which if taken properly as prescribed is no different than any other medication someone might take to help with a chronic health condition. It slightly encourages and leans into the idea that ADHD is "fixable" without medication and so taking medication is a problem (or due to a weakness, etc).

I'm not discounting the value of other forms of non-medical treatment, but in many case medication IS appropriate as part of the overall treatment that a person requires to manage their ADHD. We should not use language that stigmatizes treatment that should be between a person and their doctor.

nullpt_rs 9 hours ago

Agree with original commenter but +1 to reframing the language. I was very afraid to begin medication because I would always hear about how addicting this medication can be. But like you say, you wouldn't tell someone with diabetes that they're addicted to insulin.

paimapi 8 hours ago

jaggederest 5 hours ago

thin_carapace 4 hours ago

drakythe 6 hours ago

I think it is also worth reframing what "functioning well" means. Society expects certain things of us and our brains function in a way that those things are more difficult. Is that our fault, or society, or some mix? Isn't it possible that society itself could adjust in such a way that people who naturally struggle to sit in a cubicle doing the same thing all day every day could be offered other options? Seems like our capitalistic system that requires "productive" work as measured by those who already control the capital is maybe also a problem here.

I'm on stimulants and I agree that the post you responded to needed reframing. But I think we do ourselves and anyone else who isn't "normal" in body or mind a huge disservice just focusing on acceptance of treatment when there are larger societal changes that could be made.

OkayPhysicist 5 hours ago

schlauerfox 9 hours ago

This Doctor's videos reframed the clinical work and actual research for me, very informative and helpful to me personally. https://www.youtube.com/channel/UC0tLWu7ljYVFPiZQfHjTMsA

1659447091 5 hours ago

> it's a serious dysfunction [...] treated ADHD can make you productive "all the time", you end up addicted and pretty much permanently require drugs to function well

On its own, its not "a serious dysfunction". Within the context of operating within the current societal setup for a paycheck that provides basic survival needs, yeah.

It allows me to sit down and do 'something' without being taken completely off task for the rest of the day because someone kept knocking on the door or reading a hostile/complaint email. What that 'something' is is not always 'productive' but now I can do it without everyday things or upcoming events throwing me in a completely different direction.

Meds helps with the alignment issue. If you are taking meds and its making you "productive all the time" and you find yourself addicted to it, I suggest talking to your doctor about that. I had to work with CBT just to form a habit of remembering to take the medication, and it took a few years before I finally started to remember regularly. I still forget on days that dont fit into that routine

0x457 6 hours ago

> > you end up addicted and pretty much permanently require drugs to function well

No? First, even when I was taking it almost every day for three years, I kept forgetting to take it. Today, my last refill was in February this year.

I recommend against raw-dogging reality with ADHD without meds. At least so you have baseline "this is what normal feels like"

poopmonster 4 hours ago

I think it's important to take it case-by-case. Many people with ADHD find stimulant medications positively life-changing. However, plenty of people with ADHD also have a substance use disorder. Sometimes medication is an effective treatment for both issues; other times it exacerbates the latter.

Consider the subreddit r/stopspeeding, a recovery community for people addicted to stimulants. Many, many users share similar stories of being prescribed a stimulant medication for ADHD, then beginning to abuse it. Many of these people really do have ADHD, but find that their lives are better without stimulant medication.

zenoprax 5 hours ago

> it's a serious dysfunction and it's sad to see that it's being romanticized lately in social media or plainly discredited by some people/media

The author of the article is contributing to this by using their medical diagnosis to "outsmart" the system for writing material. Presenting a proper ADHD assessment as merely an online test just adds fuel to the fire (and if this is the only interaction they had I am skeptical of the assessment procedure).

> and while treated ADHD can make you productive "all the time", you end up addicted and pretty much permanently require drugs to function well

That's an unfair and sweeping characterization of the immediate therapeutic benefits and long-term health outcomes of individuals.

nullpt_rs 3 hours ago

Why assume malicious intent? I am well aware that the test isn’t just some Math.random() and even includes quotes from the official site that talk more about how their scoring engine works.

I don’t think I was trying to “outsmart” anything, I’m just a naturally curious person

zenoprax an hour ago