• 0 Posts
  • 19 Comments
Joined 1 year ago
cake
Cake day: July 4th, 2023

help-circle







  • I misunderstood what “war machine” meant.

    I heard phrases on TV like “it cost $1 billion dollars per month to keep the war machine running” and “the US captured 100,000 barrels of oil per day to feed the war machine” and I thought this thing must be some epic beast.

    The terrorists better watch out! We’re sending the war machine! It must have, like, sawblade hands and tank tracks and breathe fire and have machine gun turrets.

    Imagine my disappointment when I learned that THE WAR MACHINE was just a metaphor and not a Mecha Godzilla


  • I think I understand how it works.

    Remember that LLMs are glorified auto-complete. They just spit out the most likely word that follows the previous words (literally just like how your phone keyboard suggestions work, just with a lot more computation).

    They have a limit to how far back they can remember. For ChatGPT 3.5 I believe it’s 24,000 tokens.

    So it tries to follow instruction and spits out “poem poem poem” until all the data is just the word “poem”, then it doesn’t have enough memory to remember its instructions.

    “Poem poem poem” is useless data so it doesn’t have anything to go off of, so it just outputs words that go together.

    LLMs don’t record data in the same way a computer file is stored, but absent other information may “remember” that the most likely word to follow the previous word is something that it has seen before, i.e. its training data. It is somewhat surprising that it is not just junk. It seems to be real text (such as bible verses).

    If I am correct then I’m surprised OpenAI didn’t fix if. I would think they could make it so in the event the LLM is running out of memory it would keep the input and simply abort operation, or at least drop the beginning of its output.







  • poweruser@lemmy.sdf.org
    cake
    toAsklemmy@lemmy.ml*Permanently Deleted*
    link
    fedilink
    English
    arrow-up
    5
    ·
    11 months ago

    Disclaimer: this was a long time ago so things might be different now

    I flashed the “international version” of Cyanogenmod on what turned out to be the AT&T version of my first android smartphone (Samsung Galaxy S2) and nearly bricked it.

    The install froze but I knew I should never interrupt a flash, so I left it there for about 2 hours until the battery died. However, the flashing process also replaces the battery manager firmware, so it was not possible to charge the phone until I flashed the correct ROM, and I couldn’t flash the ROM because the battery was dead.

    Thankfully, that phone had swappable batteries so I solved it by buying a second one online (and waiting for shipping). But today’s phones’ batteries are usually soldered onto the board, so if it happened today I’d be SOL.

    I personally wouldn’t chance it unless someone with the same device confirmed it was OK




  • Hosting video content must be insanely expensive so I have some amount of sympathy for YouTube. However, ads earn such a paltry amount that I can hardly fathom how anyone can put up with it.

    The total amount of ad revenue between all sites is only like $12 per user per year. So IMO it is a net benefit to block all ads (save your sanity!) and simply pay the creators of content you like.

    Even just one $5 Patreon sub is worth more than all ads I’m likely to see so I block everything I can and IMO my conscience is clear