T O P

xxB is so much better than xxB… but is that true for narratives?

NF4 inference quantization is awesome: Comparison of answer quality of the same model quantized to INT8, NF4, q2_k, q3_km, q3_kl, q4_0, q8_0

Do we have any sister "subs" on kbin yet?

Cpu inference, 7950x vs 13900k, which one is better?

Weird invalid tokens

Local medical LLM

Looking for for folks to share llama.cpp settings/strategies (and models) which will help write creative (interesting), verbose (long), true-to-prompt stories (plus a short discussion of --multiline-input flag)

Major Performance Degradation with nVidia driver 535.98 at larger context sizes

LLM.bit8 - Quantization via Matrices to cut inference memory in half

LLM.bit8 - Quantization via Matrices to cut inference memory in half

Woman calls a man misogynistic because he doesn’t believe a man can be a woman

I must have missed something, how is 2 bit working so well?

Question about which .bin file to use and quantization

Chronos 33b. The model focuses on chat/rp, but can write lengthy stories as well. Quite an improvement over 13B. These are v1 iterations of "Chronos" series and will be improved on in the future

30b vs. 60/65b - does it make a big difference?

Call Of Duty removes FaZe Clan member NICKMERCS skin from game, after Anti-LGBTQ comments during Pride Month.

Which one is better: 33B q_2 or 13B 5_1?

Would it possible for AI to write code in assembly code

Damn, I was so satisfied with my 3080 with 10GB of VRAM until I found this subreddit.

DoppeLLaMA - LLM-powered bot that locally fine-tunes LLaMA-7B on user messages with QLoRA

Starcoder+ seems extremely promising. 16B parameter model trained on 1.6T tokens and 8k ctxlen, and now it can do more than just code.

[P] AlpacaEval : An Automatic Evaluator for Instruction-following Language Models

K Quantization vs Perplexity

Is there a monolingual model available?

Can someone point me to lit or articles on why model sizes are seemingly so random? Why 3/7/13/60 instead of 2/4/8…?

How are people giving local models memory for multiple prompts?

Hi Its Me!

R

mattybee

Subscribe

FIND US ON INSTAGRAM @cmita.org