Hacker News

Yes this is more like compression to remember and not for learning/understanding.

> The only major difference I see is that beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with.

It’s funny, because I thought you were talking about humans here when you wrote this. We know some things about how our bodies encode information that is sent to the brain, and we know some things about how neurons receive information and act on it, but after that we get too tired and give up on how the brain works and treat it like a miracle.

It’s like we desperately want to believe our consciousness is not just electrical impulses in our brain, and we want to ascribe agency and uniqueness to the physical processes going on in our head.

> beyond a certain number of transformations, people are willing to treat it as some sort of miracle, and too tired to figure out why it came up with the answer it came up with

It’s less about being too tired and more about being realistic about the limits of understanding.

Consider mass and energy flows in planet-scale systems: At some point we call these “weather” and change the tools with which we study them, but we never stopped trying to understand the phenomenon.

Agency?

What are you talking about?

I want freedom.

I want freedom to do what i want and not sitting in front of a computer and coding for some company.

Please AI lets burn down knowledge work and labor work. Lets create so much stress to our society that we start rethinking what works mean.

Lets redefine work into discovering the world again. Let people do old handcraft jobs, let them do more sports, let them read more, let them write and make more. Let them enjoy nature.

The data is the code. Training algorithm is the compiler. The weights are the byte code produced to run on the inference VM.

Yes. Is it data? Yes.

Is the distinction between "code" and "data" just someone's opinion? Yes. There is no such distinction in reality.

Is matter code? There is some sort of computation happening in space over time.

3 blue 1 brown has a great visual introduction to transformers, the heart of LLMs.

It's chapter 5. Start at chapter 1 if you want more background on neural nets and backprop.

https://youtu.be/wjZofJX0v4M?si=HFXbrB-5cArprGaU

We humans tend to chauvinism in all things (e.g. we're special, the center of the Universe, God made the universe for us, etc), no less when it comes to judging intelligence. The original story about thinking meat was written to help us out of our chauvinism; this derived story was written about weights for the same reason. Which is quite valid.

The actual counterpoint is demonstrated in _Blindsight_ Peter Watts. He makes a strong (and rather terrifyingly strong) point that intelligence is not consciousness.

https://en.wikipedia.org/wiki/Blindsight_(Watts_novel)

> I'm skeptical of AI sentience because we must do our due diligence, not because it's impossible. Skepticism is the only respectful approach because to grant sentience is a step away from granting rights.

Thanks for saying this! It amazes me to witness so much pushback (in HN of all places!) for the call for skepticism and scientific rigor on claims made by business which have vested interests in hyping things up.

People argue for AI sentience from a place of emotion couched in logic. they _desperately_ want it to be true and will not take a logical step back. Any argument comes back to "well doesn't a human brain work like that?" Or some variation of it.

My personal theory is a fuzzy thought about how people want to reject the concept of a higher being and want to embrace the fact that we are now able to create our own consciousness and religion is dead.

I don't understand why, but it is the undertone of every argument I've seen that is pro-AI-is-sentient, like some big unspoken elephant-in-the-room.

I would rather just judge this tech on its own merits.

edit: this comment got 1 upvote literally as I submitted it. I know @ doesn't work, but @dang, something seems very strange about that.

You could absolutely write a compelling story about a sentient toaster; it's been done before [1].

That is entirely separate to whether or not it would be a meaningful way to understand the world; a convincing story is not the same thing as one that is true.

[1] https://en.wikipedia.org/wiki/The_Brave_Little_Toaster

Well..

[0]: https://en.wikipedia.org/wiki/Tsukumogami

Only because you haven't met toasty yet.

https://youtu.be/LRq_SAuQDec?si=LFJMXZ2yGu4fxXzL

It’s reductio ad absurdum. No one cares about teapots in space either (Russel).

> Not sure which rhetorical trick is that. The point of the story, as I read it, is the technical insight (and some social implications of it).

Take a simple mechanism which has exceedingly low number of inputs and states and create a narrative around it to convey it as intelligent.

For a toaster, I can rewrite the think as "They're made of metal strips!", pointing out that their thermostat is a bimetal strip, and extrapolate from there.

I can even write one about a ruler, if I can bend it enough, no pun intended.

> Not knowing the original story, this reads fine on its own.

Yes. Because it's heavily based on the original story. The existence of the original story is kind of a critical piece here.

>But it doesn't really give a reason to accept the analogy, it just asserts it.

It's not a paper or a proof. It's a story. Doesn't want to prove the analogy, it wants to convey it.

The bit that lost me quite early in the piece was

>"A side effect. You're asking me to believe in sentient weights."

Huh? Did I miss that logical jump? Genuine question, maybe I'm not clueing into something here.

I didn't read it as coming to the same conclusion as the original, because the meat story presupposes that we who are meat already know that the aliens are wrong. (Maybe that's a humanist reading of the original, but okay). I didn't read this one as trying to make a case that we are fools for assuming that matrix multiplication can't be intelligent... I think its point was that it can't be intelligent, and that people trying to judge it the way mechanized aliens would judge meat creatures just makes them sound ridiculous.

I genuinely thought this one was a satirical take on the narrow-mindedness of the aliens in the original, even though the story tries to paint humans as narrow minded. I guess this fundamental human trait to believe that their cognition is the ultimate way to think in the universe ironically leaked into all these stories as well. Real spacefaring civilisations would probably have seen all kinds of intelligence rise from sufficiently complex systems.

It lets us consider the relativity of viewpoints. The point of the original isn’t just how humans might be viewed from the perspective of aliens. It’s that the apparent strangeness of something depends on the viewpoint and on expectations. The present story is not supposed to stand on its own like the original, it’s supposed to be considered in light of the original. In the original, thinking meat appears strange to the aliens because of their viewpoint and expectations. The present story thus points out by analogy that thinking LLMs might seem strange to us merely due to our viewpoint and expectations.

I see people give too much importance to specific engineering design choices of the current generation of LLMs. Tokenizer is not an absolutely essential part of the system. It’s just and adapter for text input/output. It can be eliminated completely and model can use bytes directly.

I think the short story captures this well. Weights (connections) are the essential and philosophically important part. They do the thinking, memory, singing etc.

> Mathematical Operation tables are not a language.

Not a natural language, but they are certainly a language as in a symbolic representation of information.

A language is a set of sentences.

A sentence is a finite sequence of symbols drawn from an alphabet.

In this sense, mathematical operation tables are absolutely a language. As are natural languages.

The Chinese alphabet is very much a dictionary. All the major tokenizers are far larger.

https://rationalwiki.org/wiki/Fractal_wrongness

I'm kind of stunned that someone is using my work to tell me I'm wrong. I wrote the code for the dish brain pong and encoding information was a huge part of what that experiment was about.

So when I way that the grok paper and the pong paper fundamentally agree I have some idea of what I'm talking about.

Comparing the tokenizer to sensory processing is a great analogy. That's exactly what your visual cortex and initial layers of the language center are doing: decoding visual representation of text into the internal neural representation.

It's a learned mapping from one representation to another, not some semantic lookup against an exogenous source.

As opposed to integers in normal programming.

The 'magic' in weights is that the rules are spread through the whole model and you can't point to one place which encodes them.

The grokking paper shows that this stops being the case with enough training data and enough compute.

Not at all. A brain is interesting because it is the computer, memory, and weights all in one. A GPU is just the calculator.

You can't move your mind to and any other brain, but weights can run on any GPU.

A tokenizer is a deterministic string-matching program, it's not made out of weights in the same sense as a neural network itself.

This way of thinking can only explain externally-visible parts of consciousness. It does nothing to address internal experience of being conscious and qualia. I don't think the internal experience has any bearing on physical reality (P-zombies would act the same externally) which makes this something outside of the realm of currently understood physics.

Oh, that's a very interesting hypothesis!

Not sure about taking it down to the level of consciousness, but makes sense regarding the sense of self, the conceptual experiencer, the perceived center of experience. It agrees well with the observation I have made again and again they my sense of self is much stronger when I'm around people, and stronger still when I'm in a context where I don't know people and/or am uncertain in social rules.

This can be as immediate as dancing in a club, and closing my eyes I feel open, free, still, the body just flowing, then opening my eyes and feeling the cage of categorization of the world, relating my self to other people as a major function, coming right back.

Also being alone in nature for me makes the sense of self drop. Without intention, spending even just a few hours alone in a forest seems to quiet down the part modeling my self in relation to the world so much. There's no need for it there. I'm not a person in a forest; I become the trees, the birds, the rustling of the leaves, the sun shining through the canopy.

There's a really, really big gap there. It reminds me of South Park's underpants gnomes.

1. Simulate others' thinking and feelings.

2. ???

3. Consciousness!

Why would one lead to the other? How would one lead to the other?

> We literally spend almost all day, every day, thinking about ourselves, in terms of our inner self.

> That is consciousness.

So thinking is consciousness?

Can there be consciousness without content? E.g. can I just be conscious of being conscious? If so, consciousness cannot be defined as the thing(s) we're conscious of.

I think this is exactly it, but let me ask another question (which is not rhetorical, I really don't know). Does the fact that one can describe what consciousness is and where it came from in humans help them to detect it in non-human and/or non-biological entities?

>consciousness is an emergent property

You would really like Michael Pollan's latest book [1], entirely devoted to his exploration of consciousness researchers' POVs on this exact topic.

My favorite quote is that ~"perhaps Descartes was only half-wrong when suggesting I think, therefore I am; it seems rather closer to I FEEL, therefore I am."~

[1] <https://www.amazon.com/World-Appears-Journey-into-Consciousn...>

----

I've grown thousands of plants; I've read two of the author's other books devoted to plants; in this book Pollan makes compelling arguments for plant sentience (over a much-longer timeframe).

Sure, perhaps plant consciousness is a bit of a stretch, but they're certainly intelligent and curious creatures. He makes both arguments supporting plant volition.

----

If you haven't seen My Octopus Teacher (Netflix), do. I'm a bald 275lb bluecollarguy... and I wept/awed (both). So beautiful, we bundled neurons.

----

Bonus quote ~"color is where reality and magic appear as-if together"~ [color isn't real, but is perceived]. We most-often see what's most-predictable, not necessarily what we actually detected [in the case of color: nothing but nanometers].

Just to be sure: The "neurons" in today's AI have nothing to do whatsoever with real neurons.

What we can do is simulate very simple brains by simulating relatively few neurons as they appear in worms. In this sense we are multiple magnitudes away where the increasing complexity implies exponential increasing difficulty.

I would think we are so far away that there will be unknown unknowns we encounter on the way.

Personally I'm not a fan of the emergence story, for a number of reasons. First is, it doesn't really make sense for consciousness to have emerged gradually through natural selection. When we and animals are conscious, the whole brain is coordinated and works to for example turn off consciousness when we sleep. And as someone else mentioned, animals with much fewer numbers seem to have a similar consciousness to us.

If consciousness really evolved gradually, you would expect to see for example dogs or gorillas having less of it, but if they has less of it, why does it function the same way? Like for example animals can be scared, happy, anxious etc, they can experience the full range of emotions and thoughts, so their conscious experience seems just as rich as ours. What I mean by this is, if you can be "less conscious", then what does that mean _exactly_? Is it that you have less content in consciousness, or is it that you feel more like you are asleep? Or something else? We don't have any examples in animals of "less conscious", I would argue.

This makes me think that rather than having emerged gradually, evolution found a mechanism by which consciousness exists, and then some animals have that mechanism and others don't. I think that if it is a mechanism, then this mechanism is located in one part of the brain, not many parts functioning together (though one possibility is that this mechanism coordinates brain activity in such a way to enable consciousness).

Our machines won't have biological systems driving their needs which in turn fuel behaviors like desire and planning for the future. They may imitate them but it won't be innate.

Surely the number of neurons is not the whole story. Human brains have approximately 86 billion and are almost definitely conscious, but there are many other animals with a lot fewer (gorilla: 34 billion, dog: 2-3 billion, guinea pig: 240 million) which appear to be conscious also.

https://en.wikipedia.org/wiki/List_of_animals_by_number_of_n...

This is not meant as a gotcha, I am genuinely curious how you believe consciousness can be an emergent property. I assume you don't believe consciousness is a physical property in the brain, so what entity is actually experiencing that consciousness? Or, what does it even mean to experience consciousness? Or are these not even the right questions?

sibling discussions are taking this as human perception. But it’s easier to think of it literally. Time is change. Physical state change. By “nothing interesting” - interpret it literally . if “nothing happens” then there is no time because there is no change, there’s no reference to distinguish each frame of time.

This is actually something I was always confused about. If nothing interesting happens as we get old, it should be boring and as result, slow slog. Yet it feels like time accelerates as I get older.

> My dog doesn't speak any language,

If you'll allow me to interpret "speak" to include "understand", I will respectfully add a contradictory note. My dog has a vocabulary of at least dozens of words and understands them remarkably well. For example, different areas we can go to have different names and saying one of them gets her to make an immediate hard turn.

I would also argue that dogs have a gesture and body posture based language they use among themselves. They, like most other animals, are not able to make the variety of noises we humans make, so they use movement instead.

I personally can easily believe that self-awareness/consciousness and language are both near-unavoidable side effects of emergent complexity, and exist in degrees across nature.

Indeed. For some reason I can shut off my thinking instantly by holding my breath, and then I feel like a very conscious but unthinking animal.

I find that a lot of arguments against emergent consciousness seem to just come out of an atheist rephrasing of abrhamic priors about the existence of a "soul". In personal chats, I've found people from East Asian countries (minus Korea, which makes sense) to be much more open to the idea of machine consciousness.

People shouldn't need to make an argument against consciousness. The responsibility is on the people claiming there is consciousness. Nobody can actually give a decent "why" for why it should be conscious, it's just pseudoscience and wishful thinking. It's not up to everyone else to try to prove a negative just because people have gotten a little too attached to ChatGPT

I could say something like "the reason why people and animals are conscious is quantum mechanical effects". Ok, maybe, it would be hard to prove me wrong because nobody knows, but it's not a very useful statement by me if I can't tell you why you should believe me on that.

I don't find the Chinese room compelling, since it appeals to intuition where our intuition is already not trustworthy. It's like trying to use intuition to understand quantum mechanics; you can't.

How do you actually know the Chinese room isn't conscious? It's merely obvious that it isn't, but that's not evidence.

I don't know if I can trust someone's understanding of arguments against consciousness as an emergent phenomenon, when he didn't even understand what the Chinese room was all about in the first place.

(Spoiler, it was not about consciousness)

> remove a brain from a body and it is no longer concious either

What is no longer conscious, the brain? Or the body? Or some other entity?

If consciousness is weakly emergent, how do we know it emerges from the solely from the brain and not, say, brain + body? Or brain + body + or environment. Or from the universe itself?

My best guess as a noob is that the vector spaces allow for unbounded contextualization. As long as the training set is large enough, it can 'infer' anything.

Proofread has a spot in that space, and layers allow patterns like terminology consistency to be expressed so your query will now tap into a subspace that will infer tokens based on whatever consistency patterns were ingested with proofreading texts.

If time dilation is said to being a product of computation, why is it that anaesthetic drugs that are taken not to the point of actual unconsciousness cause it. Dont anaesthetics sort of shut everything down/inhibit all that kind of cognitive activity (compute?)

I never said it wasn’t fair use, I just think it’s kind of disrespectful of the author’s wishes. Being legal doesn’t make it moral or admirable.

It seems to me that since the advent of image generators, art has been firmly defined by artists to mean that it was made by a human. But there might be a spectrum of human involvement where the less a human is involved the less it's art.

What happens too often during these discussions is that someone who writes "make me a cool image" gets conflated with someone used ai to fixup a small rock in their natural landscape drawing. (two extreme ends)

One problem though, is that we don't really know how much the supposed human author was involved in the piece. Now that it's becoming hard to judge, people against ai art can proudly change their opinion on on a piece once they learn that it was made by ai. I've come to think this is somewhat respectable, like you see a video of some extraordinary event (before ai) and then you learn that it was fake, just for views or something.

But on top of all this, there are different ways to "consume" art. Artists may think more about who the artist is as a person and what they felt when they made the piece, while non-artists may just enjoy the piece for what it is, detached from the artist. These two perspectives clash a lot.

Obviously not. And neither is this newfangled "digital art" thing.

It's not real art unless you used a brush or a pencil for it, and no, "PC Paintbrush for MS-DOS" really doesn't count.

There's two meanings to "the body is a complex machine" and I think you're missing the forest for the trees here.

1) The abstract "dictionary" version: It'd be technically correct to say that the body is a machine under the definition of "A machine is a thermodynamic system that uses power to apply forces and control movement to perform an action.".

2) But there's also the less abstract/technical: "The body is alike the complex machines we have built", and this is much less true. Especially for the brain. The "neuron" analogy in machine learning is effective, but entirely wrong; We do not fully know how even a single neuron works, nevermind any complex system made out of multiple of them.

With regard to AI, there's a lot of people extrapolating "There is no magical animating spirit, the brain is just a pile of stochastic molecules following the laws of physics" into "The brain is an inert pile of matter, computers are an inert pile of matter, ergo AI/LLMs are like the brain!"

Especially so by people who have a financial/legal interest in doing so. "AI is just like a brain, fire your employees and buy our LLM now!", "AI is just like a brain, so it's totally not copyright infringement!"

I find the claim subjective experience may be illusory absolutely baffling. I can only speak for myself with certainty, but I am entirely sure I have subjective experience. All the other propositions I believe could be false but I don't see how I can be wrong about experiencing something. I could be a brain in a vat (or weights on a GPU) and be specifically programmed to only come to false beliefs and still I can be sure that there is an experience I am the subject of. I cannot provide empirical evidence for my experience, that is why it is subjective. I cannot be entirely sure you are experiencing anything, and when I encounter people who don't share the same baseline intuition here I do begin to wonder if this is truly a universal across humanity. But I can't think of any other assumption which I would be more comfortable as a foundational axiom other than, "I am experiencing something." I do not require additional evidence for it because I experience the truth of it directly

I think the truth lies somewhere between these two extremes. An LLM is not a human brain, and does not try to emulate one. It should not be a surprise that an LLM does not behave like a human brain. So we can not infer things either way. The best we can say is that an LLM appears to exhibit very similar behavior to a human brain, under certain constraints. So maybe we can infer that the human brain has something in it that operates in a similar way to an LLM (like the human "unconscious", or "intuition" maybe). It seems obvious to me that a human brain and an LLM are not comparable things, for many reasons. So we can not make inferrences one way or the other.

Interesting, so you would say that your experience is .. illusory? In what medium exactly? Illusion requires a substrate of some kind. "Awareness"? What's that?

Neurons are themselves things we experience (indirectly). Once seen through a microscope or known about in some fashion the only way they "exist" is by you having the experience of knowing them. It's not the other way around. One thing is more fundamental here. What is this experience? What are the atoms of this? "Atomic particles"? How would you even approach an answer if your building blocks are themselves part of what needs to be explained?

The hard problem cannot even be touched if you start out like this.

Refuting the "subjective experience" axiom does not lead to 'any "subjective experience" is completely tied to neurons', you also need to explain why the subjective experience is tied to neurons. And that's precisely what computational theories of mind do not account for: the link between subjective experience and neurons. I am not arguing that neurons (or the brain) are not a necessary condition for subjective experience.

It’s the opposite.

Descartes made clear that subjective experience is the ONLY thing we know. Everything else is theories to explain the phenomena we subjectively experience.

We theorize that there is a physical world and other beings like us having similar subjective experiences, because that seems the best explanation for our subjective experiences. But we might be living in the Matrix, with all the people we think we are interacting with and just sophisticated simulations.

> at requires explanation beyond the material, that is axiomatically assumed without evidence

Nobody talked about anything out of neurons. The question is still open.

"that is axiomatically assumed"

passive voice doing a hell of a lot of work in this phrase

> Do you disagree with the assumption that cells are machines? They seem pretty machine-like to me. You just said it: they are machine-like. No, cells aren't machines for two reasons: 1) machines, by definition, are artifacts created by human beings. 2) The nature of a living organism is completely different from that of 'machines'. (even if we a re able to replicate a cell in a lab, like the group from Craig Venter did). Autopoiesis being one very big difference, another being emergence-within-the-environment (life) vs design-conditioned-by-will (machines)

> ... if the mind is an emergent phenomenon from machines (cells) then it seems quite likely that a mind could emerge from other, different machines. Since cells cannot be defined as machines, the argument about mind emerging from machines does not hold.

It's not a huge leap to imagine biological cells might have a rudimentary consciousness

Pan-psychists might argue that your subjective consciousness is an aggregate of all the cells/molecules, etc in the system

"While each biological cell operates largely on its own chemical cues, they all coordinate through complex nervous and chemical networks to create your unified, subjective experience."

You might be a rigid materialist

> I certainly don’t think individual cells have any subjective experience or sense of agency.

There's definitely research and scholarship that would beg to disagree with you there. At least in terms of completely writing off the notion of "agency" when it comes to cells.

Dr. Michael Levin's lab is doing some pretty cool work. https://drmichaellevin.org/

> Do you disagree with the assumption that cells are machines?

Yes? Literally no machine ever built by humans is capable of (or even hinting at beginnings of capability for) replication or novel synthesis like cells are, let alone autonomously, it’s quite unconceivable that anyone would take this to be a reasonable assumption in the first place.

> If we agree weights producing text may emerge consciousness, given large enough, then DBs must've gained it long ago.

If we agree that silicon can perform calculations, then beaches must have been working out log tables long ago.

> Modern psychology more or less has settled around the idea that consciousness is a product of the body.

The kind of consciousness we know. Jumping to the conclusion that that's the only kind possible, or even stronger that the way ours evolved is the only way this could have happened, is completely invalid.

> It's raising the possibility that our thinking could be modeled as a machine. Given that, as you acknowledge, we don't have a clue how our mind works, it seems premature to rule out this possibility.

In order to raise the possibility that our thinking can be modeled as a machine, there needs to be a previous question: 'can our thinking be modeled at all?'. And before that, already the idea of possibility: 'our thinking could be something that can be modeled'. Since we know 'we can think', asking 'can machines think?' needs the assumption that machines and brains are alike. If there is no assumption, then we should ask 'if brains and machines are alike, then we could raise the possibility of thinking a machine could also think'. But when we say 'brains and machines are alike', we are implicitly saying 'brains are like machines'. There is no problem asking 'can machines think?', but there is indeed a problem with implicitly assuming that brains and machines are alike when we do not known it yet. I am critiquing the idea of assumptions here, not the research.

Indeed, scientific type thinking like that where you ask a question that you can do experiments on to see if machines can pass the test is probably the way to progress. The philosophical waffling mostly just goes in circles.

To investigate consciousness you'll probably get further trying to build conscious machines with agency and comparing them to biological ones than reading Heidegger.

None of the works I mentioned solved or try to solve the mind/body problem or the issue of consciousness.

They are a frame of reference for not stepping into the common fallacies that the AI research field is based on.

> I highly recommend the philosophers read some neuroscience Philosophers do read neuroscience and technical reports on AI.

> The whole "model weights" thing in AI is modeled after the synaptic connections and between actual neurons In reality, it is a really poor and basic model of what is actually happening in a real brain

Brains and modern AI systems (LLMs for example) are structurally different. (Don't get confused by topology. A structure is more than topology: it is also what the structure is made of, thus the properties of the material contribute and define what can emerge atop the structure)

Hard disagree. We only discovered the role that glial cells play in processing around 2014. We're still uncertain how patterns of activation consolidate through long term potentiation, let alone how signaling encodes information. We understand quite a bit about the role of the hippocampus and subiculum in encoding memories; but we don't understand the structural layout of engram complexes - which were themselves mapped for the first time only in 2022!

Taking effective results in machine learning, and somehow assuming that they apply to cognition - simply because neural nets were inspired by our limited knowledge of neural signaling and structure - is like trying to apply aircraft engineering to studying ornithology. For a better articulation of this point (from the reverse direction) check out the paper 'Could a Neuroscientist Understand a Microprocessor?' from 2017 - https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...

> There are also differences between discrete neuron firing and weights as signals, but there is enough similarity to make artificial neural nets useful and do things similar to what real one do.

There is barely a surface-level similarity. The best example I can come up with is this…

Imagine the most intricate and beautiful tall building that you can think of. Think like an older skyscraper in Chicago or a palace. There are water features and moving parts everywhere but also tiny little handmade carvings and materials throughout.

Now imagine we have no reference designs and no blueprints - we hire an architect to attempt to study the building by looking at it from a distance and understand everything they possibly can about it. She can go into the building to check but every time she does, it stops functioning normally.

That architect is a neuroscientist.

Then the ML researcher is like a graphic designer who sees the work that the architect is doing and makes a napkin sketch of the building the architect has been studying, to use for a project later. Sure the designer has some of her representations. But the difference in complexity between the designer’s napkin sketch and the architect’s analysis is massive. Several orders of magnitude.

Then another many orders of magnitude is the level of detail that the architect can understand about this strange building without being able to fully interact with it, versus the actual complexity of the building.

So yeah, an AI is modeled after neurons in the sense that they represent a couple of surface level features of neurons. But the difference in complexity is about as much as a napkin drawing of a grand building represents the actual structure and details of the building, no matter the level of skill that the graphic designer has

> On the contrary, I highly recommend people in Philosophy of mind and linguistics should start reading AI research papers because their theories and ideas are highly outdated, even ancient. Your books are from 1927 and 1972 respectively and Turing's article is from 1950s. And they are relatively new with respect to other works in Philosophy.

People in philosophy and cognitive linguistics do read AI research. Don't get fooled by the publishing dates: although Heidegger's work dates from 1927, the work is contemporary. The same happens with Dreyfus' work. Again, publishing dates don't mean anything here. Maybe you can clarify why they are outdated.

> If one doesn't adequately understand what we have in 2026, how can they theorize about it? As others they don't understand how the mind/brain work, BUT ALSO they don't understand how the AI works.

I would say that people involved in the critique of AI do know how it works. But I've found that is normally the case that people in AI research does not have the framing provided by works in philosophy or cognitive linguistics.

> I think philosophy people and Linguist will catch up in a century, like they did with Turing. Philosophers of this century are not in humanities or literature. They are in science and engineering. What do you base your claims on? Plenty of philosophers work in humanities, literature, sociology as well as science and engineering. Philosophers not catching up? The critique on automation and AI already dates from the early 20s if not before.

> Heidegger was trained on priesthood and Theology. You should read greater minds like Hinton, LeCun etc. if you want to think on these things. They are the real Philosophers.

Sorry, but this does not make too much sense. Hinton and LeCun are great in their own fields. But seriously, they are not philosophers, they are inventors.

Yikes dude. You may need to expand your horizons a bit.

Indeed, I know there is research going on placing agency and free will as illusions of our own subjective experience. Yet, there is a big gap from a purely mechanistic view of the mind to the idea/phenomena of subjective experience.

we know the brain does not use gradient descent. (i agree it doesn't matter)

I like Leibniz's comparison of the brain to a mill, which he invokes in an argument[1] that seems to be the predecessor of the modern "hard problem:"

> supposing that there were a mechanism so constructed as to think, feel and have perception, we might enter it as into a mill. And this granted, we should only find on visiting it, pieces which push one against another, but never anything by which to explain a perception.

[1]: https://en.wikipedia.org/wiki/Leibniz%27s_gap

Vegan ML engineer here. In total agreement with you. People are just moving the goal post to keep themselves protected from the obvious conclusion: there is nothing really all that special about us humans. Perhaps subjective experience is simply the internal state of a self supervised continuous learning algorithm and we don't like that conclusion very much.

“Reflections on trusting trust” is the paper that posits a compiler which is edited once so that when it compiles a program, it adds a security vulnerability to it, and when it compiles it’s own source code, it adds this edit into itself. Then it is used to compile its own source code once. Then this edit is removed.

Now any study of the program or compiler source code will not show any vulnerability, but compiling the program will make a vulnerable program, and recompiling the compiler from its clean source code will not fix the situation.

This carrying down of a pattern which is not written down anywhere, a flaming torch lighting a torch lighting a torch, is analogous to four billion years of life on Earth. We talk like DNA is an everything-code that defines a human and a human brain, but it’s the implicit behaviour of cells (‘compiler’) and the mechanisms inside them which interpret DNA. The unbroken chain of life getting more and more complex and never being restarted from scratch, with the behaviours not written down anywhere for us to study. How does DNA arrange for x, y, z to happen? Maybe it doesn’t at all.

Accidentally stumbling on a mechanism that is simple enough to be recreated with every human birth might be possible, accidentally stumbling on a mechanism that took evolution billions of years to find and which it has hung onto by copying it and has never recreated it from scratch, could be much less likely, in a much bigger search space.

I don’t think the OP is discussing mind-body dualism. There are many materialists who believe that consciousness maybe a non-computational process.

But it is worth pointing out that something like 80% of the world (it fluctuates depending on the survey but its around that) believes in some non-physical spirit, life force, or soul.

It’s a very HN bubble thing to start a discussion with the assumption that everyone must be a materialist.

Not OP, but I have read Being and Time on and off over the years, and every time, I am blown away how precise Heidegger describes things and the being of things. Granted, I am German, so I can directly read the source without needing a translator. That might change things.

Regardless, I think Heidegger gave one of the fullest metaphysic-free accounts of the human experience and what Being is. And he starts from scratch. You essentially can read him without having to first study the whole western continental philosophy and he will construct the whole system by himself. Tremendous work

I would say that the LLM is something completly different especiayll as its not a normal algorithm but is very close to what brains do.

It came out of nowhere. It’s all emergent. I’m convinced this is possible with just about anything given enough data. We will be seeing a near magical physical outputs LLM in the near future. It’s going to take in video and sounds and spit out physical movements that will be just as mind blowing as when 3.5 came out and it will come out of nowhere.

great perspective. makes you wonder how we will innovate onto this.

Yep.

At least change the link at the top of your blog post to the author’s website: https://terrybisson.com/theyre-made-out-of-meat-2/

Your modern adaptation is perfect for now-common explainers [this time IS different; it's not programming, it's weights]; these "just analogues" will be the thing I show everybody first whenever discussions of consciousness/AI come up (then will play Jon Benjamin reading original).

Bravo. Really helps (even with my own) perceptions of newness. Similar to stsitned short-story (on dentists, backwards).

I am making a narrower claim than the Chinese Room argument. I actually see consciousness via mechanistic processes as entirely possible.

My point is about current LLMs specifically, which the article is clearly referencing. For a present day transformer, I can write down in my notebook everything the model ever sees as input, plus weights and architecture notes, and can compute the next token with pen and paper, just extremely slowly.

This does not prove that a computation cannot be conscious. But if the same transition from prompt to next token can be decomposed into an explicit sequence of arithmetic operations, then the burden is on the defender to explain: Where, in this process, consciousness is supposed to enter?

Mind that the Chinese Room experiment is comparing the mind of a Chinese speaker to a different mechanistic symbolic procedure. I am, however, executing the exact same mechanistic process, whether it is done on a GPU or with pen and paper.

My hope is that some magical consciousness process emerging from electricity circulation or whatever people believe the mechanism of consciousness would be in the case of LLMs obviously becomes implausible, unless you hold a particularly strong form of substrate-independence, stronger than what most substrate-independence supporters would need to accept.

Although I get that it's a metaphor, I really dislike the "some people are born with a GB300 while others have an L40S" part. Even as a joke, it ranks people like hardware tiers, which is dehumanising and uncomfortably close to eugenic language. On top of that, the analogy also breaks down if you try to implement it literally.

For an LLM, you have clear stages of mostly feedforward computation over finite numbers and a perfect way to reconstruct the computation.

For meat, even if you model it under a purely Newtonian approximation, you need to simulate at least the immediate closed system around it which is continuous, thermodynamic, chemical and so on. You'd need to choose an arbitrary time step and update enormous amounts of coupled physical state to get an inexact simulation of a minimal slice of reality.

You would have a much harder time obtaining even a substrate-independent dead organism, comapred to LLMs that are already substrate-independent, which is basically what my notebook example shows.

You're basing that on..?

It’s just a matter of perspective. Procedures and memory are the same, and they’re also different, depending on how you want to look at it.

I was referring to transistor base bit - the way it 'switches' the circuit on/off. That bit is the primordial creator of 'logic', IF branching, compute and the intelligence.

The concept of "flow" is questionable though. Bubbles in water move upwards, but it is actually water that is flowing downward around the bubbles. Just because bubbles do not contain water, we can't say bubbles are not flowing.

When it comes to electrons and positive charges, their material existence is equally non-physical. Actually, none of them might be "flowing", as the concept of flowing applies only to physical things that occupy some spatial volume and spatial location.

Not especially. Depending on where you set your standards for "holding a conversation" you can satisfy the requirement with a classical markov chatterbot, a well-trained parrot, a copy of Eliza, or a telemarketer flowchart drawn on a sheet of paper. Only the markov bot is made out of "weights" in the sense of a statistical model.

Parrots are intelligent animals, albeit with a limited capacity for vocabulary and syntax compared to a human, and Eliza and the flowchart are made out of explicitly encoded rules and conversational tactics.

>I mean, Blake Lemoine went crazy

Ah, the unsung AI psychosis[1] pioneer.

[1] https://news.d.umn.edu/articles/expert-alert-ai-psychosis-20...

Ahh thanks. I didn't follow the article links.

Your EDIT is untrue, but thanks for linking to originals.

Here is Jon Benjamin reading Bisson's original text: <https://www.youtube.com/watch?v=5usXhX0zaO4>

Video adaptation.

https://youtu.be/T6JFTmQCFHg

> Do those actually qualify as alien, if they're products of our human culture and just the substrate is different?

I'll posit "alien" is a spectrum.

For the sake of the argument, let's assume that some form of panspermia is real and the same tree of life has reached Earth, Enceladus (moon of Saturn) and TRAPPIST-1 (a different solar system in our galaxy). Let's also say there was a second abiogenesis event somewhere in Messier 104 (another galaxy).

Earth to Enceladus would arguably be already "alien", but there might be similarities, maybe there was something there that looked like one of our Archaea, while sharing none of our Eukarya and having its own domains of life.

Earth to TRAPPIST-1 would be distinctly alien, evolved so differently it'd be almost unrecognizable, but they'd likely still be carbon-based lifeforms sharing the same basic building blocks. Maybe something like lipids forming cell walls would also be seen there, but they'd likely be independently evolved.

Earth to M104, any similarity would be at best convergent evolution. Truly unarguably alien.

https://web.archive.org/web/20180423171909/https://cosmosmag...

>> While you write very dismissive and pseudo philosophical

Even with the "pseudo" in front, I'm very sorry any of my writing sounds philosophical; I didn't intend that sort of confusion :-). The "dismissive" is not exactly intended either; instead, I was aiming for "bitter".

>> enough people do not believe

Here we believe different things. First, enough people, even today, do believe. Second, the body of culture we are raised in accrued during centuries. The vast majority of it comes from people who believed. Everybody in my family was atheist and yet I was raised homophobic, and I have it from good sources I'm not an isolated case.

>> I'm a complex biological thing.

That state comes with a big wallop of misery. For millennia, we have used faith to justify that misery. Not a year ago, I was at the hospital, next to the bed of a dying girl. Can't forget the doctors saying "we do what we can, but we are not here to prevent what is going to happen." Coming from them, it was sensible resignation. Sensible because as long as we believe those things are inevitable and there's nothing we poor humans can do, we can absolve ourselves.