LLM API Recommendation

loweendspirit202310

Hello everyone,

I'm working with a nonprofit organization looking to use LLM APIs for translating and rewriting public domain books. We've explored options like TogetherAI, Anyscale, and Replicate, but their pricing (around $0.8-$1 per million tokens) is a bit steep for our budget.

Does anyone know of more affordable API options for accessing models like Llama 3.1 70B that would be suitable for our translation project? We're open to lesser-known providers or academic partnerships that might offer discounted rates for nonprofit work. Currently using loopin with their $50 free credits (Using code 5tSu) but we need much more volumes.

Any suggestions or experiences to share would be greatly appreciated. Thanks in advance for your help!

LowEnd

If you're talking about old public domain books, chances are there's also a public domain translation available. That would yield way higher quality translations, and lower the cost drastically.

loweendspirit202310

Thanks for helping. We rewrite the story for 8-12 year old kids, breaking it into engaging chapters that are easy to follow. We'll also incorporate interactive elements, allowing young readers to converse with the book's characters.

adly

Perhaps OpenAI GPT4o-mini using the batch API is an option? ($0.075 / 1M input tokens, $0.30 / 1M output tokens)

Encoders

try deepinfra, I've been using them for 70B models but unsure if it's cheaper than you mentioned

havoc

Use openrouter.ai then you can try various.

If you need cheap then deepseek is probably your best bet, though don't think it is particularly well suited to translation

loweendspirit202310

@adly said:
Perhaps OpenAI GPT4o-mini using the batch API is an option? ($0.075 / 1M input tokens, $0.30 / 1M output tokens)

@havoc said:
Use openrouter.ai then you can try various.

If you need cheap then deepseek is probably your best bet, though don't think it is particularly well suited to translation

@Encoders said:
try deepinfra, I've been using them for 70B models but unsure if it's cheaper than you mentioned

Thank you for all the info!! We'll test above mentioned providers. GPT4o-mini is an ideal option. I'm not aware of their batch API before. For real-time inference, we'll use OpenRoute to evaluate various LLM models. So far, Llama 3.1 has proven capable. We tried DeepSeek's service, but despite being cost-effective, its performance fell short of expectations.

rockinmusicgv

If time isn't a factor you could run GPT4All or llama.cpp locally

loweendspirit202310

@rockinmusicgv said:
If time isn't a factor you could run GPT4All or llama.cpp locally

Don't have local GPUs.. API works better for us