LLM ("AI") discussion

2»

Comments

  • bikegremlinbikegremlin ModeratorOGContent Writer

    @somik said:

    @bikegremlin said:
    We could make a similar discussion for human artists too.
    AI does do it on a lot huger scale though (can't be compared), for profits (mostly, in one way or another).

    Many renaissance era artists copied each other for profit as well. Back then, they were "inspired" by each other and sold their painting for money. AI is just the new version of that...

    AI is the difference between hand-copying a whole book for day - and sharing a digital copy on torrent to millions of folks in one day.

    Doesn't come close.

    Likewise, unless they fix the source quotations, it will be fun (AI written information, answers, and innevitable errors with no source quotes to correct them or the AI).

  • havochavoc OGContent WriterSenpai

    Kinda curious which country moves fastest to adapt to this entire change. Clearly the genie is not going back in the bottle so we're going have sizable chunks of professions that are going to need a plan B. And not sure how flexible the average ~45 year old is.

    I'm younger and would definitely struggle to pivot, especially if I need to maintain same income. In fact 99% sure I couldn't

    Thanked by (1)bikegremlin
  • bikegremlinbikegremlin ModeratorOGContent Writer

    @vyas said:

    @somik said:
    Anyone manage to run AI at home with any sort of boost on amd gpus?

    Many ways to skin this cat ..

    a. Try LM Studio : https://lmstudio.ai/models

    I would start with the lightweight models- Gemma3 4 B ? Phi4 etc,

    b. Or, get the desktop app for Huggingchat (basically Chrome) or

    c. Install AnythingLLM with Gemini or Groq via API (not the X/Twitter Grok)

    There was some discussion about pros and cons of Anythingllm's "Propereitary" licensing model, in one of the discussions here. You may have to look it up .

    Best wishes

    Here's my intended use case - looking for recommendations (to save time and effort on trial and error):

    Needs

    Copy/paste all my articles, markdown notes (Deathnotes), and some forum posts (mostly text only with a photo here and there).

    Get "my AI" answers for stuff like:

    • "Where did I write about mounting paste's effect on tightening torque?"
    • "What do I think about mounting paste's effect on the optimal tightening torque?"
    • "How does mounting paste affect tightening torque?"

    Again, the answers should come from the pre-fed notes, so a closed system (no garbage-in, like ChatGPT has).

    Hardware

    Ryzen 9 5900 (12-core), 64 GB RAM, Radeon RX 6800 graphics card (16 GB VRAM), 2TB SSD and 5+ TB HDD storage available.
    Windows 11 Pro at the moment.

    What i figured

    I should be able to comfortably run a 13B LLM - and it should suffice for this - correct?

    Any recommendations for the model and configuration?

  • havochavoc OGContent WriterSenpai

    @bikegremlin said:

    Anyone manage to run AI at home with any sort of boost on amd gpus?

    Many ways to skin this cat ..

    a. Try LM Studio : https://lmstudio.ai/models

    I would start with the lightweight models- Gemma3 4 B ? Phi4 etc,

    b. Or, get the desktop app for Huggingchat (basically Chrome) or

    c. Install AnythingLLM with Gemini or Groq via API (not the X/Twitter Grok)

    There was some discussion about pros and cons of Anythingllm's "Propereitary" licensing model, in one of the discussions here. You may have to look it up .

    Best wishes

    Here's my intended use case - looking for recommendations (to save time and effort on trial and error):

    Needs

    Copy/paste all my articles, markdown notes (Deathnotes), and some forum posts (mostly text only with a photo here and there).

    Get "my AI" answers for stuff like:

    • "Where did I write about mounting paste's effect on tightening torque?"
    • "What do I think about mounting paste's effect on the optimal tightening torque?"
    • "How does mounting paste affect tightening torque?"

    Again, the answers should come from the pre-fed notes, so a closed system (no garbage-in, like ChatGPT has).

    Hardware

    Ryzen 9 5900 (12-core), 64 GB RAM, Radeon RX 6800 graphics card (16 GB VRAM), 2TB SSD and 5+ TB HDD storage available.
    Windows 11 Pro at the moment.

    What i figured

    I should be able to comfortably run a 13B LLM - and it should suffice for this - correct?

    Any recommendations for the model and configuration?

    Yeah should be fine though you'd need to quantize it a bit to make space for context. I'd probably do qwen3 models. Either dense one or you can try the MoEs 32B and put it on system mem partially. Will still be fast given low activations in the MoE

  • NeoonNeoon OGContent WriterSenpai

    New Deepsex just dropped.

  • havochavoc OGContent WriterSenpai

    @Neoon said:
    New Deepsex just dropped.

    Now I just need 20 grand worth of GPUs to run it...

  • @somik said:

    @bikegremlin said:
    We could make a similar discussion for human artists too.
    AI does do it on a lot huger scale though (can't be compared), for profits (mostly, in one way or another).

    Many renaissance era artists copied each other for profit as well. Back then, they were "inspired" by each other and sold their painting for money. AI is just the new version of that...

    And given the profit it generates, it's here to stay, whether we like it or not.

    Forgive me if I'm wrong, but I'm not aware of any AI outfit actually generating anything that could be called 'profit'. Right now it's just a bunch of techbros throwing money at it and hoping it becomes profitable.

  • havochavoc OGContent WriterSenpai

    Deepseek claims they're profitable, so while there is lots of free VC money shenanigans going on I don't think it's true across the board.

    That said...deepseek is in China...who knows what sort of indirect subsidies (elec price etc) they didn't count in their claim.

  • They claim a theoretical operating profit, which is both a dubious claim in itself and ignores virtually all costs so ... yeah, it's mostly marketing bull to back up the assertion that their models are cheaper to run.

  • @ahnlak said:

    @somik said:
    Many renaissance era artists copied each other for profit as well. Back then, they were "inspired" by each other and sold their painting for money. AI is just the new version of that...

    And given the profit it generates, it's here to stay, whether we like it or not.

    Forgive me if I'm wrong, but I'm not aware of any AI outfit actually generating anything that could be called 'profit'. Right now it's just a bunch of techbros throwing money at it and hoping it becomes profitable.

    There are AIs running in the background of most social media and online shopping websites, monitoring your preferences. You dont interact with it but it interacts with your data, processing it and optimizing the recommendations so suit your preferences.

    AIs are being used by many security companies to scan and track people to predict behaviour and alert the security team of any potential deviation from the norm.

    Those were generic examples. And if you prefer specific examples,

    1. Amazon – AI for Recommendation Engines and Logistics
      Application: Personalized product recommendations, dynamic pricing, and supply chain optimization.
      Profit Impact:
      The recommendation engine drives 35% of Amazon’s total sales.
      AI-driven logistics and warehouse robotics have significantly reduced operational costs and delivery times.

    2. Netflix – AI for Content Personalization and Production
      Application: Tailors user content recommendations and predicts successful content.
      Profit Impact:
      Netflix estimates that its AI recommendation engine saves the company $1 billion per year by reducing churn.
      AI analytics help greenlight original content with higher chances of success (e.g., House of Cards was chosen based on viewer data).

    3. Tesla – AI in Autonomous Driving and Manufacturing
      Application: Full Self-Driving (FSD) features and factory automation via AI.
      Profit Impact:
      The FSD package sells for $8,000–$15,000 per vehicle, a high-margin software product.
      AI also increases manufacturing efficiency and quality control.

    4. Google – AI in Advertising (Google Ads) and Search
      Application: Smart Bidding, ad targeting, and search ranking algorithms.
      Profit Impact:
      The majority of Alphabet’s ~$300 billion revenue comes from ad products, where AI optimizes targeting and bidding.
      AI helps advertisers get better ROI, making the platform more attractive.

    5. Zara (Inditext) – AI in Inventory and Trend Prediction
      Application: Demand forecasting, inventory optimization, and design choices.
      Profit Impact:
      Reduces unsold inventory and boosts profitability by predicting demand more accurately and minimizing waste.
      Enables fast turnaround in fast fashion, maintaining Zara’s competitive edge.

    6. Spotify – AI for Music Recommendations and User Retention
      Application: Discover Weekly, Daily Mixes, and personalized playlists.
      Profit Impact:
      Personalized playlists increase user engagement and retention, critical for subscription revenue.
      Reduced churn helps Spotify maintain long-term profitability from recurring subscriptions.

    7. JP Morgan – AI for Document Review and Fraud Detection
      Application: COiN platform (Contract Intelligence) automates document review.
      Profit Impact:
      Saves 360,000 hours of legal work annually, reducing operational costs dramatically.
      Fraud detection AI reduces losses from cyber threats and fraud.

    Thanked by (1)bikegremlin

    Never make the same mistake twice. There are so many new ones to make.
    It’s OK if you disagree with me. I can’t force you to be right.

  • Not sure any of those are LLMs (which was, I thought, the topic).

    And let's be honest, they were labelled 'expert systems' a decade ago, and only got rebranded as "AI Inside!" when it became the new sexy.

    (that also reads like LLM output, but I can't decide if you just asked Claude to make your argument for you, or if you're just being ironic :lol: )

  • havochavoc OGContent WriterSenpai
    edited May 29

    @ahnlak said:
    Not sure any of those are LLMs (which was, I thought, the topic).

    And let's be honest, they were labelled 'expert systems' a decade ago, and only got rebranded as "AI Inside!" when it became the new sexy.

    (that also reads like LLM output, but I can't decide if you just asked Claude to make your argument for you, or if you're just being ironic :lol: )

    They're all different things. Haven't found a nice diagram that has an overview, but best as I can tell it's like this:

    Neural network (broad overarching category)

    • Graph neural nets --> The recommendations somik listed
    • LLMs (but even this seems to come in diffusion flavour and tensor flavour)

    Expert systems are the thing that came before Neural Nets. Basically the expert systems approach that prevailed, then was abandoned when they discover just throw enough data in and you can skip trying to hardcode expert knowledge. Then the LLMs started hallucinating, then they started talking about "grounding" them in truth...aka here is a known good dataset basically full circle back to expert knowledge set lol

    I find the whole field quite confusing since it moves so fast & often one model has different things at different layers

    Thanked by (1)bikegremlin
  • @ahnlak said:
    Not sure any of those are LLMs (which was, I thought, the topic).

    And let's be honest, they were labelled 'expert systems' a decade ago, and only got rebranded as "AI Inside!" when it became the new sexy.

    (that also reads like LLM output, but I can't decide if you just asked Claude to make your argument for you, or if you're just being ironic :lol: )

    Why not both? Using LLM to generate replies seem to piss @bikegremlin off :lol:

    In terms of LLM, it's just the next version of "AI" that we have been using for a while. The current usage for LLM is to replace tech support with LLM. Another "profit" generation is to write up real looking spam emails and scaming people through it, while email providers are using LLM to scan your emails to protect you against spam and "log" your preferences so you are more susceptible to advertises.

    @havoc said:
    They're all different things. Haven't found a nice diagram that has an overview, but best as I can tell it's like this:

    Neural network (broad overarching category)

    • Graph neural nets --> The recommendations somik listed
    • LLMs (but even this seems to come in diffusion flavour and tensor flavour)

    The examples I listed almost all use tensor flow, usually powered by a nvidia GPU. GNN is only used for large data crunching, not behavior analysis. For example, if you want to train a AI to learn to recognize pictures of fruits, you usually use tensor flow. LLM is just a type of model trained using tensor flow and Natural Language Processing (NLP).

    @havoc said:
    Expert systems are the thing that came before Neural Nets. Basically the expert systems approach that prevailed, then was abandoned when they discover just throw enough data in and you can skip trying to hardcode expert knowledge. Then the LLMs started hallucinating, then they started talking about "grounding" them in truth...aka here is a known good dataset basically full circle back to expert knowledge set lol

    I find the whole field quite confusing since it moves so fast & often one model has different things at different layers

    LLM = NLP + tensor flow
    They train the models using a LOT of text and then let users interact with the AI model to train it. AI will always generate an answer. Issue is that it may or may not be correct. So it needs the user feedback to improve the model, and slowly it'll train it's model to be as accurate as possible.

    Training AI models is fun! I remember using Convolutional Neural Network (CNN) using Python TensorFlow to train a model to recognize fruits... here's my github if you want to play with it!
    https://github.com/somik123/python-cnn

    Never make the same mistake twice. There are so many new ones to make.
    It’s OK if you disagree with me. I can’t force you to be right.

  • vyasvyas OGSenpai

    Interesting discussion on how hugging face makes “money” while hosting so many llm models.
    Hint: they don’t. Yet have insane valuation
    https://x.com/levelsio/status/1928030106401861958

  • @vyas said:
    Interesting discussion on how hugging face makes “money” while hosting so many llm models.
    Hint: they don’t. Yet have insane valuation
    https://x.com/levelsio/status/1928030106401861958

    https://huggingface.co/pricing

    Probably similar to how dockerhub earns money.

    Never make the same mistake twice. There are so many new ones to make.
    It’s OK if you disagree with me. I can’t force you to be right.

  • NeoonNeoon OGContent WriterSenpai

    @havoc said:

    @Neoon said:
    New Deepsex just dropped.

    Now I just need 20 grand worth of GPUs to run it...

    8B Distilled is up too.
    Runs even on a IPhone

    Thanked by (2)vyas bikegremlin
  • NeoonNeoon OGContent WriterSenpai

    The new Deepsex is lit.
    I ran "What is 1+1?" and it basically called me stupid.

    The amount of FPS we are getting though, even in loading screen, on CPU, is intense.

Sign In or Register to comment.