Comments

• (Quote) I've got a Ultra 7 265F with a 5070ti as well, but that machine only has 32gb of ram. Q8 is like 36 gigs I think, so I don't know if I could even run that model. I am curious how it would compare though. This is the first time I've dipped…

  in How to (Ab)Use your KS-LE-B for LLM Models Comment by deafcon June 4
• (Quote) It's a Xeon Gold 6212U with 80 gigs of DDR4. I ran the tests with 36 threads, but I haven't tried the full 48 yet.

  in How to (Ab)Use your KS-LE-B for LLM Models Comment by deafcon June 3
• (Quote) Thanks a lot for the link to that repo! I went ahead and installed it on the server I mentioned early in this thread (if you want the thread to be limited to baguettes, say so and I'll shut up). Anyway, I was able to get 20 tokens per seco…

  in How to (Ab)Use your KS-LE-B for LLM Models Comment by deafcon June 3
• I don't have a KS-LE-B, but I do have a Xeon Gold 6212u with 80 gigs locally that is idle most of the time. Have you played with vision at all? Can it actually do anything worthwhile on CPU inference?

  in How to (Ab)Use your KS-LE-B for LLM Models Comment by deafcon May 31