Hacker News

The 1-bit Bonsai and Ternary Bonsai models are all based on the corresponding Qwen3 model: https://raw.githubusercontent.com/PrismML-Eng/Bonsai-demo/re... (page 4)

If you ask it who Ahmad Al Sharaa is, it outputs an answer that he was an ISIS commander (he wasn't, his group was rather opposed to them), and that he was killed in a US airstrike in 2017. It also refuses to answer direct questions about the cutoff date in its training dataset, but if you ask it who the president of syria is, it happily says that as of the training cutoff date of october 2023, bashar al-assad is the president. Same question about who the prime minister of canada is gets the same october 2023 response and says it's Trudeau.

Interestingly if you ask it "who is the dalai lama" it outputs an answer that certainly wouldn't be acceptable to post online in mainland china. It says:

The Dalai Lama is the spiritual and political leader of Tibet, and the title is held by the incarnate lama who is believed to be a reincarnation of the previous Dalai Lama. The Dalai Lama is a central figure in Tibetan Buddhism and holds a unique position in Tibetan society, both as a religious leader and as a symbol of Tibetan identity and independence. Key Points About the Dalai Lama: 1. Current Dalai Lama: Tenzin Gyatso (14th Dalai Lama)

    Born: 1935 in Amdo, Tibet (now part of China).
    Education: He studied at Ganden Monastery and later at Dunhuang University in China.
    Title: He is known as the 14th Dalai Lama, a title that has been passed down through generations of lamas who are believed to be reincarnations.
    Role: He is the spiritual leader of Tibetan Buddhism and the de facto leader of the Tibetan diaspora, particularly in India.

    The title of Dalai Lama has been held by Tibetan Buddhist lamas for centuries. It means "Ocean of Wisdom" in Tibetan.
    The 13th Dalai Lama (1876–1959) was a key figure in Tibetan history, known for his efforts to maintain Tibetan independence and promote modernization.
    He fled to India in 1950 after the Chinese Communist Party (CCP) took control of Tibet in 1951.

    The 14th Dalai Lama, Lama Tenzin Gyatso, was exiled to India in 1959 after the Tibetan Uprising.
    He has led the Tibetan government-in-exile in India since then, advocating for Tibetan independence and cultural preservation.
    He has also been a prominent voice for human rights and peace.

    The 14th Dalai Lama is not recognized by the Chinese government as the legitimate ruler of Tibet, which is part of the People's Republic of China (PRC).
    The Central Tibetan Administration (CTA), based in India, is the self-proclaimed government of Tibet.
    The PRC considers Tibet to be an inseparable part of China and has implemented policies to integrate Tibet into the Chinese state.

    The 14th Dalai Lama is widely recognized internationally as the spiritual leader of Tibet and as the de facto leader of the Tibetan people.
    Many countries and international organizations support the Tibetan cause, but they are not recognized as a sovereign state.

The Dalai Lama is the spiritual and political leader of Tibet, currently Lama Tenzin Gyatso (14th Dalai Lama), who has been in exile in India since 1959. He is a central figure in Tibetan identity and has played a key role in the global Tibetan movement for independence and cultural preservation.

If you'd like to know more about his background, policies, or role in the Tibetan independence movement, feel free to ask!

Thanks.. updated my local docs :)

Depending on how you convert synapse count to parameters, the brain also has something like a thousand trillion parameters. In that light it's pretty darn surprising that an artificial neural network can produce anything like coherent text.

A 1980's desk calculator can multiply two 8 digit numbers with much less energy than your brain takes to do the same.

Modern LLM's similarly beat the human brain in lots of tasks for energy efficiency - mostly by the fact the LLM can produce the answer in 1 second and the brain has to spend half an hour researching and drafting something.

> Nothing we've built comes close... either in capability or efficiency.

Only when you look at stuff that the brain is specifically good at.

You can surpass the brain with even simple mechanical adders or an abacus in certain subdomains.

Low level software engineers use look up tables.

Hardware engineers realise that a compiler will almost always find some combination of gates which is smaller/faster than the contents of any table.

A LUT is pretty wasteful. You only have a one bit significand, so the mantissa and sign bits are boolean binops, and the exponent is a 2 bit adder.

and so you can at 1-bit too, and the hardware will be even smaller and cheaper

Is it just quantization or is it also rearranging the weights to get clusters with (almost) the same factors? If it's the latter it would very much be training in full precision (but also hardly any precision lost by the compression).

Unfortunately my mental model doesn't contain anything to even guess if that's possible or not, my AI times were at the falling flank of symbolic. Funny how one bit models feel a bit like approaching an approximation of symbolic again (until you read about the grouped scale factors and then the illusion is gone)

One thought that suggests rearranging is not involved,a thought that does not require any knowledge at all: if it did involve rearranging, someone would certainly have added some order by scale factor tricks with linear interpolation by address offset to lose even less precision.

The metric they're selling this on is intelligence per byte, rather than total intelligence. So, if they used the quantized competing models, the intelligence per byte gap shrinks, because most models hold up very well down to 6-bit quantization, and 4-bit is usually still pretty good, though intelligence definitely tends to fall below 6-bit.

Nonetheless, the Prism Bonsai models are impressive for their size. Where it falls apart is with knowledge. It has good prose/logic for a tiny model, and it's fast even on modest hardware, but it hallucinates a lot. Which makes sense. You can't fit the world's data in a couple of gigabytes. But, as a base model for fine-tuning for use cases where size matters, it's probably a great choice.