A 3D fruit fly on macOS desktop powered by the real FlyWire connectome (github.com)

295 points by phoenix120 13 hours ago

causal 12 hours ago

I appreciate the open source approach over the startup's sensational claims. This is more transparent.

That said, I think it still presents itself as if the fly is being controlled by the connectome, when (if I'm reading the source correctly) it's more accurate to say scripted behaviors have been hooked up to be triggered by the connectome.

Put another way, you might as well have the "escape" trigger open a YouTube video of a fly flying away. That would probably be less misleading too.

dullcrisp 9 hours ago

Yeah I don’t think it’s clear if the connectome is modeling real fruit fly behavior or is just being used as an entropy generator.

londons_explore 4 hours ago

If the connectome could be wired to individual muscles, and then demonstrate the ability to walk and fly that would be amazing

ramraj07 12 hours ago

I would argue that this is pretty disingenuous (as is most connectomics research I suppose) since its implying we have a fully wired connectome of the fly thats actually meaningful.

squibonpig 11 hours ago

Why isn't it and what do they have? I'm not really informed on this stuff.

melagonster 9 hours ago

andbberger 6 hours ago

cursor input get scripted into loom input to the LC4s which are evidently simulated faithfully enough to drive giant fibers, which is actually how fly escape behavior works. what more could you want

WithinReason 21 minutes ago

Related (fictional):

https://qntm.org/mmacevedo

MMAcevedo (Mnemonic Map/Acevedo), also known as Miguel, is the earliest executable image of a human brain. It is a snapshot of the living brain of neurology graduate Miguel Acevedo Álvarez (2010–2073), taken by researchers at the Uplift Laboratory at the University of New Mexico on August 1, 2031

DaiPlusPlus 12 hours ago

I have no exposure to connectomes - or really anything quite like this (as opposed to good ol’ fashioned perceptrons) - so forgive my following uninformed question:

Can someone explain if this software is ethical?

jephs 11 hours ago

If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work.

If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're like just fruit flies you know?

yawpitch 6 hours ago

If your circle of empathy doesn’t include fruit flies, endeavor to expand your circular of empathy until it does.

Goto 0.

ivell 5 hours ago

sgt 5 hours ago

firtoz 4 hours ago

janalsncm 11 hours ago

> Can someone explain if this software is ethical?

Probably not. People have been coming up with ethical frameworks for thousands of years and we have yet to come to a consensus.

So it’s very unlikely that someone can say conclusively whether this is ethical or not, because that would require us to either 1) prove a single ethical framework is correct and then apply that framework to this software or 2) prove that under all possible ethical frameworks this software is either ethical or not ethical.

_def 11 hours ago

Genuinely curious, is there like a simple list of competing ethical frameworks?

janalsncm 11 hours ago

jasonjayr 11 hours ago

viccis 11 hours ago

Honestly pretty funny that you gave a (fairly useless) metaethical answer instead of responding to the actual meaning, which was clearly soliciting ethical judgments about it.

randomImmigrant 7 hours ago

Yes. Because it’s a simulation of a living thing. It’s still electrons moving around silicon. It deserves the respect your MacBook does. Which is not nothing. But certainly not what’s due to a life.

reverius42 6 hours ago

A lot of people believe that this reality, in which we're conversing on HN, is also a simulation (thus making the fruit fly a simulation within a simulation).

lp4v4n 10 hours ago

How could this be anti-ethical? You're just running a very complex algorithm in your machine.

j16sdiz 9 hours ago

When one truly believes:

0) Subjecting fruitfly life to this kind of treatment is unethical.

1) Life is just physics in action.

2) Physics are just algorithm (or our physical world is a simulation)

It is not hard to understand why they think it is unethical.

My question are

1) is all life equal?

2) is all suffering unethical?

3) is this world materialistic or are there something out there?

4) should a list of items begin with 0 or 1?

fluoridation 8 hours ago

vincnetas 6 hours ago

Is ethics another tool in a power play game (religion being another) that people use to exert dominance over other people? No one expects of cat or pigeon to be ethical. But somehow humans sometimes get overloaded by external demand to think of gazillion things that might offend someone (sometimes unreasonable demands, or how it might get interpreted (sometimes incorrectly)) before acting.

Im not saying ethics is bad, but sometimes people bend to to extremes (same like spirituality or religion)

yawpitch 6 hours ago

Ethics has nothing to do with how an action is perceived, rather its focus is on is the action moral.

One should never take any action without considering the ethical implications of that action. Being capable of that is, arguably, the sole thing that meaningfully separates us from the (real or simulated) fruitfly.

vincnetas 5 hours ago

fluoridation 5 hours ago

bad_username 5 hours ago

I believe that any simulation of consciousness is consciousness, so ethical implications apply. But I also don't believe fruit flies are conscious for any definition of that word.

ramraj07 12 hours ago

Why do you think it wont be?

twothreeone 11 hours ago

Well, one obvious question is if the simulated brain experiences pain, or more generally can it suffer? And if so, how would we tell.

fluoridation 9 hours ago

logicallee 8 hours ago

hellohello2 9 hours ago

Yes I can, no problem. First figure out how consciousness works and tell me, then I'll be able to explain if the software is ethical or not. Its simple really.

DaiPlusPlus 2 hours ago

As we don’t know yet, it has to be determined by the balance of probabilities; but even if the weight of evidence leads us to conclude that-there’s-no-consciousness going on when you run this, there’s still going to be a chance of us being wrong, let’s say 10%… Is the utility (…or entertainment value) of this program worth a 10% chance that we confidently, yet unwittingly, create perhaps the most cruellest machine it’s ever possible to make, and then have to live with ourselves?

andbberger 5 hours ago

unbelievable amount of coarse graining to get from a real fly to this. same leve of ethical quandary as giving your dog a stuffed bear to tear apart. besides, you should see the things we do to real flies...

a1o 12 hours ago

> What's modeled vs. measured

> Honesty section:

I wouldn’t mind an human written README

rossy 8 hours ago

> Honesty section:

This one gets to me. Not only is it useless (paragraph would have the same meaning if it was removed) but wdym "honesty section," Claude? The whole readme should be honest! One of the many causes of psychological damage when reading AI-generated technical writing is the reminder that one of its tics is that declares when it's being honest or real, casting doubt on the rest.

isoprophlex 4 hours ago

Don't you ever add a few dishonest paragraphs to your docs to spice things up?

egorfine 3 hours ago

Honestly? I don't even know how to write anymore. Generative text is unequivocally nice, polished and correct. And it so happens that I do write in the same style as well. I never use LLMs to write human content, but here we are: I cannot use "honestly" anymore. And the emdash. Thank God we got "delve" back, though.

dag11 9 hours ago

> runs as a 1 kHz leaky-integrate-and-fire (LIF) simulation

That hyphenated thing is always such a give. Clearly a thing discussed in the Claude conversation, made its way into the plan, then the readme. It always creates hyphenated phrases out of things it feels are core requirements.

I'm actually over caring about Claude's obsession with the concepts of "real", "honest", and "rather than". What annoys me the most now is how every single sentence, bullet, etc that even makes its way into the products it builds are advertisements for itself. Product settings pages now read like ads, or like something written by a student begging for extra credit from the teacher. You can't unsee it!

etienne89 7 hours ago

To be fair, as a term it has always been written as "Integrate-and-Fire" neuron model. In this case correct would be "Leaky Integrate-and-Fire" but that is nitpicking.

Resources that outdate ChatGPT: - W. Gerstner "Neuronal Dynamics" (2014): https://neuronaldynamics.epfl.ch/online/Ch1.S3.html - E. Izhikevich "Which model to use for cortical spiking neurons?" (2004): https://ieeexplore.ieee.org/abstract/document/1333071

vhantz 12 hours ago

> live *spiking* simulation

aperrien 13 hours ago

Interesting work! There's NeuroMechFly (https://github.com/NeLy-EPFL/flygym/) available, you might try using that to simulate the fly body in real time as well.

pavel_lishin 12 hours ago

Ah, Mr. Acevedo, nice to see you.

emsign 14 minutes ago

Fall; or, Dodge in Hell comes to mind

uniq7 13 hours ago

No demo video?

davidkwast 13 hours ago

I just found this video on YouTube now:

https://youtu.be/A8fDr7Rr7yM?is=cHwOQVrli9oXLdqY

yreg 12 hours ago

That doesn't show this project.

hmokiguess 12 hours ago

Open to see what it was about, stayed for the banger soundtrack

tedd4u 11 hours ago

FYI it takes less than 30 seconds do clone, build, and run.

asimovDev 5 hours ago

it would've taken the similar amount of time for the OP to do a 10 second screen recording and paste it into the project root to be linkable from the README.md and it would avoid thousands of people having to waste compute cycles and electricity and 30 seconds of their lives cloning and building , wouldn't it?

UltraSane 11 hours ago

If you have a mac. I don't.

qingcharles 11 hours ago

At some point in the future would it be possible to simply download a DNA sequence of an entire animal as a "ROM" and boot it into a virtual "reality"?

What detail level of reality would we have to emulate for an egg to grow into an adult animal?

Leynos 10 hours ago

To give you a sense of the computation needed, a spatiotemporal physics simulation of a minimal (493 gene) bacterium's full 105 minute cellular cycle took 4-6 days of GPU hours on two A100s.

https://www.sciencedirect.com/science/article/pii/S009286742...

Compare this with a c elegens nematode: ~20000 protein coding genes and 959 cells, with all the extra machinery you'll find in a eukaryotic cell.

I'm reading this as both a question of raw simulation surface and combinatorics.

fooker 9 hours ago

We'd need analog solutions for this simulations, and slowly these solutions will start to resemble biology in a dystopian way.

lp4v4n 10 hours ago

Would we be able to have a "digital human" when we reach the level of creating this kind of brain mapping for the human brain?

Honestly I think we're still discovering how powerful neural networks are and certainly they can do computations that sound very close to intelligence in some contexts, but I still find it hard to think that we can simplify the whole reality of an incredibly complex chemical machinery down to a bunch of matrices multiplications.

fooker 9 hours ago

> bunch of matrices multiplications

What these matrix multiplications are doing is approximately solving pretty high order differential equations.

The approximation is not very analog, sure. But neither is nature.

neuraldenis 3 hours ago

Hi, im the author of this repo, thank you for posting here!

t00 3 hours ago

Is there a chance for a Linux version?

sgt 6 hours ago

This has to be some of the coolest stuff I've seen on HN in a while. I'm running 3 flies on my desktop right now, all powered by realistic brains.

jochak 2 hours ago

quite interesting, very small neural nets mimicking the brain of a fruit fly

ecto 9 hours ago

Things like this are why I come to HN! Cool hack :)

make_it_sure 4 hours ago

does it simulate hormones as well?

kypro 11 hours ago

Oh nice, might use this to create some art for my home.

https://www.instagram.com/p/DMfi-9Vsl-u/

> Infinite Pain is a computer running a deterministic neural simulation of a microscopic nematode worm’s brain (c. elegans), subjected to regular pain stimuli.

ddlsmurf 10 hours ago

are you ok ?

_def 11 hours ago

theodric 4 hours ago

It's cool and all, but I have real ones in my kitchen, and I'd prefer to keep the computer room free of them if it's all the same to you.

actionfromafar 13 hours ago

Now this is real AI.

bitwize 13 hours ago

This is straight up ghost-dubbing.

DonHopkins 4 hours ago

I made a housefly simulator on the Mac once, using a visual programming language called Bounce / Body Electric, developed by Chuck Blanchard at VPL Research, originally used for VR simulations with the DataGlove and Eyephone and a pair of SGI workstations to render each eye.

https://en.wikipedia.org/wiki/VPL_Research

The simulated fly's purpose was to annoy the simulations of Rush Limbaugh and Jesse Jackson by landing on their heads, who were watching TV and arguing about the closed caption stream. I wrote to Jaron Lanier about it 27 years ago:

    At 12:31 AM -0400 7/8/99, Hopkins, Don wrote:

    Hi, Jaron. We've met briefly once or twice - I'm a friend of David
    Levitt's.

    What's this about Sun acquiring the rights to the VPL patents, and
    Body Electric?
Jaron Lanier's VPL and Body Electric page:

https://web.archive.org/web/20051217153424/http://www.advanc...

>VPL had problems maintaining quality in our hardware- though I still sometimes find our headmounts and gloves in use. But our software was really great and can still offer a lot. A challenge to VRML people: Don't you DARE turn out more VR tools that aren't at least as good as this old stuff!

    Does Sun actually have the Body Electric source code? What version do
    they have?

    Does anybody at Sun even know how to compile a Mac program?

    Last time I worked there (admittedly a long time ago), they were too
    embarrassed to allow Macs in the building (which might make somebody
    realize how bad Unix sucks in comparison).

    Has anyone tried to rewrite it in Java?

    When I was working with David at Levity and Interval, I totally
    overhauled the source code to Bounce: porting it to the latest version
    of CodeWarrior and the PowerPC, cleaning up C code translated from
    Pascal, that no human had ever touched before, and just generally
    re-indenting and adding white space so it was pretty to look at.

    Then I implemented a new interface for plugging in DM's, based on
    ActiveX (yes, ActiveX runs on the Mac).  I added a new data type that
    you can flow along blue wires: a COM object (aka a plug-in ActiveX
    object in a shared library).  

    Then we made plug-ins modules that produced and consumed the new
    plug-in data types, like strings and polymorphic dictionaries.  With
    these new data types, we were able to model very complex simulations
    as nested trees of dictionaries, strings and numbers, treat
    dictionaries as high level objects, pass them all around on wires,
    reading and modifying them at will.

    The thing that the original Body Electric was missing is a way to
    dynamically model structured data like Lisp s-expressions and
    association lists, which is now possible in Bounce, using
    dictionaries. (The swivel3d trees just don't cut it for representing
    "knowledge".)

    Bounce still had the "M4" Director player rendering engine, but it
    didn't do everything we needed, so I implemented my own graphics
    library for drawing sprites, playing sounds, and stuff like that.  We
    used it to implement a simulation of Rush Limbaugh and Jesse Jackson
    watching TV and arguing over the closed captioning stream.

    A big fat housefly would skitter around the screen, land on their
    faces, and tickle them into waving their hands around and
    pontificating.

    I just got a 400 mhz G3 powerbook, and fired up a 2 year old copy of
    Bounce and the crazy Limbaugh/Jackson demo, and it still works,
    actually like a bat out of hell! I ran into David and John Szinger
    (another Bounce programmer from Interval) a few days ago at a 4th of
    July party, and they really got a kick out of seeing the old demo that
    we worked together on, still running!

    I live in Oakland just south of Berkeley. Drop me a line if you're in
    the bay area, and would like to see what Bounce has evolved into.

        -Don

    From: Jaron Lanier
    To: Don Hopkins
    Sent: Thursday, July 08, 1999 6:13 AM
    Subject: Re: Body Electric lives?

    Hey there, and thanks for writing!

    Yup, Sun owns VPL and Body Electric, and my guess is that if anyone
    looked very closely it would turn out Interval doesn't have rights to
    Bounce.  But no one is likely to look very closely, so let's forget
    about that.  (I don't think Sun is aware of Bounce or the work at
    Interval.)

    I had never asked what was done with Bounce at Interval- it's
    fascinating to hear what you were up to.  I live in NYC for the most
    part, but I'd love to see it sometime when I'm in the Bay Area.  Also
    Chuck Blanchard lives in SF and you and he might want to trade demos
    sometime.

    There IS a community of Body Electric users.  It is STILL building the
    most interactive 3D virtual worlds of any tool (though Alice, from
    Carnegie Mellon, is the other hot contender).  That's SHAMEFUL!  While
    BE sucks in every other way, all the more recent vr design tools,
    especially the vrml ones, simply avoid the problem of deep
    interactivity. How could the community be so whimpy, at this late
    date?

    On the Body Electric side of things, there has also been some work
    updating to CodeWarrior/PowerPC (shame we didn't share that work!), as
    well as support for Quickdraw3D, OpenGL on the Mac, OMS, and some
    other standards.

    There have been some changes to the interface, but not as ambitious as
    yours.  The neatest thing is a debugger/tracer tool that is a pleasure
    to use.  Since BE is used mostly in 3D domain, there are also some
    tools dealing with textures, lights, etc.

    I had always thought the swivel tree was ridiculous, of course, but on
    the other hand I liked the idea that the virtual world and the
    knowledge base were the same thing- that unity encourages the
    visibility and grabbability of the underlying concepts.  I think the
    brain works that way- there isn't some barrier behind which everything
    gets abstract- instead, it's user interface all the way to the bottom!
    What I think would be the coolest long-term destination of BE would be
    extending the scenegraph so that it was as powerful a knowledge base
    as you'd want...

    I'd LOVE to see a fancy BE release, in JAVA, or at least spitting JAVA
    out.  The question, of course, is where the money would come from.
    I've tried to talk Sun into an open source release so the community
    could hack on it voluntarily (the source was included in the patents
    that were granted, so it's actually already released by the US
    government anyway, though only on paper).  Unfortunately, Sun doesn't
    want to do that.  What they've said instead is that I can choose up to
    six sites with free hacking privileges at a given time.  Bizarre!
    With so few sites, I think there'd need to be money to make sure the
    sites stayed focused on it...  You'd have thought Sun would know
    better by now.

    The body electric community is surprisingly NOT entertainment
    oriented, though I and a few others still use it that way.  There are
    people hidden away who are still using it for ergonomic simulations,
    simple surgical planners (because the fancy systems are too rigid to
    model some situations), cognitive test rigs, and some work with kids.

    You should know there was some tension about BE/Bounce at one point.
    The problem was that Chuck Blanchard wasn't credited as the lead
    designer/programmer of BE/Bounce when David brought the program to
    Interval.  Chuck's name was reduced in stature in the "about" and he
    was not mentioned in some important semi-public demos at Interval.  He
    was also offered a pseudo-position to help with Bounce at Interval,
    but without health benefits - and Chuck has MS and absolutely NEEDS
    health insurance.  I at one point yelled at the guy who runs Interval
    (forgot his name..)  about their treatment of Chuck- and David is
    STILL mad at me for making a fuss about it.  But I felt I had to.

    So that's the scoop!  Both sides of the mystery revealed!

    All the best,

    Jaron
Homepage of Jaron Lanier:

https://web.archive.org/web/20050212085044/http://www.advanc...

Here are three goals for "good VR" from David Levitt, with whom I work at Pantomime Corporation, and who used to work on VR at VPL:

https://news.ycombinator.com/item?id=8023893

logicallee 12 hours ago

This is absolutely astounding. What an implementation. A connectome is like a full brain scan!

I am asking Claude to port this to the browser and I'll host it, so you can try it without installing anything.

Some details about the port: Claude is able to do porting tasks like this. I'll put up a link when you can try it in your browser. I was able to get it to port the complete BitNet b1.58 2-billion parameter model several versions ago (in February, so this would have been Opus 4.5) to browser, so it can do so. This will run on the browser. I'll have it make it so that on mobile you can tap the fly or turn on the camera and it sees you, and same on desktop actually. Since the connectome is only 668 neurons, this will run super fast. Actually I'll make it allow for several flies, and let you drop food. I did something like this before here[1] but using the real connectome is something very different and very amazing. I'll have it add an experimental 3D mode with threejs, I'm not sure if that is also possible or not, we'll have to see it. This size of task usually takes Claude a few hours, so check my comment in the morning for the implementation, I'll reply here once it's done and hosted.

[1] https://stateofutopia.com/papers/1/evolving-brains-cull-long...

mapontosevenths 10 hours ago

It's pretty amazing stuff!

I was very excited the first time I saw a connectome as well. Years ago I managed to get most of openworm running and connected to some Lego Mindstorms to let the worm roll around my house. I'm not even sure the connectome was finished at that time.

Honestly, I barely had any idea what I was doing and we didn't have AI tools so I was never really sure what I wired to what. I did manage to hook... something to the ultrasound sensor and... another thing to the wheels so that when the sensor was triggered it would back up.

What blew my mind was the realization that these weird hybrid critters were probably the first true immortals. Now that we have a full connectome, who will bother with another? Students will probably just keep instantiating copies of this one to tinker with forever.

logicallee 8 hours ago

I can't imagine why people hate my comment so much (4 downvotes), when I linked an existing fruitfly project I already did, and Claude is working on what I wrote now, just as I wrote. Maybe people thought I was being sarcastic when I wrote "A connectome is like a full brain scan!" but I wrote that since maybe not everyone knows what a connectome means.

mapontosevenths 7 hours ago

spuz 3 hours ago

dekhn 11 hours ago

It's weird to write MacOS specific software in this era.

MBCook 9 hours ago

Why? If I was writing this I would write it for my OS.

Why would I write it for anything else?

I assume that’s what the author did.

asimovDev 5 hours ago

do you mean they should've spent more time writing for other platforms or do you insinuate that they should've shipped a chromium browser instead of writing native software?

bashZorina_09 6 hours ago

Is it?

I'm observing the largest tech companies in the world release macOS/iOS-first or only, and consequently supports said version the best in this era.

DonHopkins 3 hours ago

You've curiously left out which platform it's not weird to write software for.

Is there only one platform that's not weird, or is MacOS the only weird platform?

Some of the best platforms and software are weird. Get over it.

bigyabai 11 hours ago

The unstoppable "rewrite it in Swift!" vs the insurmountable "where's the Windows version?"