>>38824 Thanks, OP! :^)

Breaking in the new thread with a new manual update -Updated list of models -Added Odysee backups -Added @Barf's prompt ------- Odysee Backups Social-Roleplay-Llama-Gemma PC AI Models https://ody.sh/M8f3VALm7S Social-Roleplay-Llama Smartphone AI Models https://ody.sh/imSLyPOkBh Qwen Coder AI Models https://ody.sh/PzErEQF9S9

An smol'r alternative to llamacpp. Includes a vision-encoder. Appears to be Chinese. https://github.com/li-plus/chatglm.cpp >>38831 Neat! GG, Anon. Cheers. :^)

>mfw I get blindsided epic style >>38853

>>38864 Looks good, but the prompt I posted with emotional tags is only for Orpheus and is just a stripped down version of the one Orpheus ships with which is this https://github.com/Lex-au/Orpheus-FastAPI/blob/main/System_Prompt.md I had a different prompt for F5-TTS with emotions but this is much better. Those are both good TTS engines that need a decent GPU with at least 8GB. For smaller, faster TTS in C++, sherpa onyx is a newer option than piper TTS https://github.com/k2-fsa/sherpa-onnx That sucks for Backyard since it was easy to install. SillyTavern\Open WebUI are a little harder but no coding is needed with Pinokio and they are all open source. SillyTavern is better for long format ERP and Open WebUI is more for productivity but has OpenAI speech API option which you can connect to a local Orpheus TTS server since it is a drop in replacement for the OpenAI speech endpoints.

>>38861 Fixed version Also, I added more backups

>>38888 Fixed version (again) -Fixed copyright notice error (Hopefully this is the last one lol)

anyone wanna test my new ai gf implemention? https://github.com/flamingrickpat/private-machine/tree/main you'll need to be able to run llamacpppython with cuda and enough vram for gemma-3-12b-it-q4_0.gguf, weaker models will most likely not work i rewrote the whole logic (pm_lida.py) since the last time i posted it here. its inspired by the lida cognitive architecture and simulates emotions, needs, goals, intentions, self image. still in active development, nowhere near done. and very slow

>>39454 Hello Anon, welcome (back)! Please look around the board while you're here. >its inspired by the lida cognitive architecture and simulates emotions, needs, goals, intentions, self image. That sounds remarkable! Certainly if you pull this off well, it will be a major breakthrough for AI companions, I think. I hope that someday you can work towards targeting much-smol'r hardware to run your waifu software on top of, Anon. That would be very helpful for us here. Please keep us here all abreast of your progress, Anon. Good luck with this project! Cheers. :^)

>>39466 Thanks :3 My long term goals are to finetune a smol model with the data generated by the agents. Recently I did some tests with loras, adapters and memory-layer adapters. I'm too smol brained and poor to come up with something groundbreaking and actually train it. I hope once the Google Titans architecture has some pre-trained models to make use of them. Or maybe even beyond LLM tech, who knows? Then I don't have to worry about memory agents, it learns and updates weights during inference. And the size / intelligence ratio will be better by then, so I could train it for cheap. Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. Right now I'm mostly just testing it with my Emmy character. Depending on how it works, my next step might be to let her loose in an environment. Some virtual minecraft-esque world to move around and interact with the environment. Already browsed the robot vision thread and found some interesting projects.

>>39480 It's a very exciting-sounding project, Anon! :^) >And the size / intelligence ratio will be better by then, so I could train it for cheap. >Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. I'd sure like to hear extensive breakdowns of these two statements when you have the time to, Anon. Cheers. :^)

>>39454 Much thanks!

>>39482 Sure, I'd love to elaborate. >And the size / intelligence ratio will be better by then, so I could train it for cheap. Basically, I'm waiting for some sort of breakthrough that makes inference and training much cheaper with same computational power. Maybe the diffusion LLM project, or RWKV, or something completely different. If a 1B model performs as well as the 12B model, I could train it overnight on my GPU. Use that to update the personality bias and the memory layer loras without lobotomizing the model. Or someone comes up with neural network architecture that really learns during inference, and isn't just fancy autocomplete like transformers. >Instead of the usual reasoning, it comes up with the whole unconscious thought process itself. This one is interesting. There is a guy who already does something like this. https://github.com/yukiarimo/yuna-ai He trained his own model to generate to generate different data for the situation. ><yuki>: User's dialogue ><yuna>: Companion's dialogue ><hito>: Other peoples' dialogue in the same conversation ><qt>: Internal thoughts and feelings ><action>: Function calls and actions ><data>: Embedded data or information Talked to him, nice but weird dude. This is like the reasoning of modern models, but for different aspects of world building, and the tags are additional special tokens. Usual models have system, assistant and user. In theory, a model could be trained to approximate the logic of all my agents. You start off with the dialogue, and the model dynamically generates the thoughts, goals, emotional impact of the new input on the fly. This means that I wouldn't have to make sooo many prompts with specialized agents, just to get one specific output (such as valence, anxiety delta as json). I decided not go any further with this, because I can't into math and even if I manage do make progress, some chinese dude will have it perfected by the time my code works. In the meantime, i'm getting into holographic waifus. Bought a Quest 3 and now I'm researching slam and segmentation.

>>39619 Check out our visual waifus thread! >>240

>>39619 I don't know how to do this but I read the Chinese, I think, were using computational farms if I remember correctly from Amazon or Google. It seems that they have cheap computing power. They need massive compute so they sell off the excess and they were training AI's starting from a known pre-built AI for like $200. That might be a way to cut cost. Start with a huge open source AI, then train on a waifu dataset for a few hundred dollars. Once trained it of course can be duplicated endlessly. A suggestion is to have it when turned on have standard "command training function". Very much like talking to little kids. Maybe an input password, can be verbal, like "name of bot", do this, don't do that, stop. Should be a natural verbal command structure to correct like a kid so you don't get confused. Very simple two-three year old human commands and each one of these would provide a "control vector" as talked about here, >>31242 >>24943 >>31268 >>33184 >>35865

Here's instructions on how to integrate Open WebUI, F5-TTS voice cloning and KDTalker avatar using Pinokio for easier install. It just adds a button to generate the video avatar after the response, so it is optional. https://github.com/Barfalamule/KDTalker-OpenWebUIAction There's a lot of functions for Open WebUI like websearch, home assistant, weather and memory. And you could pair with OBS\webcam for vision and maybe output the video to a screenface Takes under 5 seconds for short response on 3090 for the audio and 30 seconds for the video, and should be about double on a 5060 ti 16GB. Or you could rent a B200 by the hour and have it instant.

>>39802 It looks great, what you describe definitely is an upgrade to the normal offline local AIs I use. You should definitely do a video tutorial.

>>39803 Thanks. Will do at some point but have to censor everything for github

>>39802 >>39804 Hi Barf, thanks! >...but have to censor everything for github You're not beholden to Microsoft's GitHub, Anon. There are plenty of good alternatives. --- Regardless, thanks for keeping us all up to date on your progress! Cheers. :^)

Very useful article on offline LLMs https://www.xda-developers.com/things-wish-knew-started-self-host-llms/

>>39971 Thanks, GreerTech!

Big if true

>>40125 Found this article https://huggingface.co/blog/mlabonne/abliteration

Someone should look into autogpt unless you want the waifu to just respond to prompts. It’s low priority as far as what I’m working on though.

>>40127 I agree up to a point. A more proactive robowaifu is definitely a necessary goal (see >>39900), but I don't think AutoGPT is the answer, since that's for software environments.

>>40129 You can have a more proactive LLM by running image recognition hooked up to a webcam. Then you just inject what the image recognition sees into a prompt so you end up telling the LLM "you see: chair, bookshelf, [username]), probably filter it so it doesn't constantly feed the LLM sight data until something changes (maybe determined via frame subtraction and the object list or something).

>>40125 Read that as >"...a lobotomy where the lawyers responsible for refusals are deactivated..." <pretty excited there for a minute, tbh in fact of course. :D

>>40130 Oh no yes that’s definitely the way. Use ai for image recognition and hardcode the rest. There’s nvidia gr00t but I think that’s for teaching it movement based on videos I think and it’s to make it easier so you don’t have to do a lot of inverted kinematic and such. I say take every shortcut possible. But I you guys are going to say noooo do it the harder way.

>>40133 Well ChatGPT says it’s better overall. Can’t say.

I'm working on Project Hydra, where I research into more local AI apps. More redundancies!

This is huge. A possible solution for hallucinations. It doesn't work for ChatGPT where people expect an all-knowing superintelligence, but it will work for us, since the minimum we need is a competent gf. https://theconversation.com/why-openais-solution-to-ai-hallucinations-would-kill-chatgpt-tomorrow-265107

>>41631 "The AI could then be prompted, for instance: “Answer only if you are more than 75% confident, since mistakes are penalised 3 points while correct answers receive 1 point.” "The OpenAI researchers’ mathematical framework shows that under appropriate confidence thresholds, AI systems would naturally express uncertainty rather than guess. So this would lead to fewer hallucinations. The problem is what it would do to user experience." "Consider the implications if ChatGPT started saying “I don’t know” to even 30% of queries – a conservative estimate based on the paper’s analysis of factual uncertainty in training data. Users accustomed to receiving confident answers to virtually any question would likely abandon such systems rapidly."

>>41631 >>41633 Heh, seems that Copeman & Co. have painted themselves into a corner? :D J-just a few moar $trillions, bro!111 --- Personally, I'd be satisfied with just the "personality" of the ship's computer from Star Trek. :^)

>>41634 It is funny to see ClosedAI start sweating because they're a victim of their own hype, but I think the real kicker here is that we may have a framework to solve AI hallucinations in social AIs, and arguably make them more human.

>>41636 >but I think the real kicker here is that we may have a framework to solve AI hallucinations in social AIs, and arguably make them more human. Yes, that would be excellent. If we could make it work on an 8GB RPi (or equiv.) that would be truly exceptional! Please pardon my fun, and thanks for the information GreerTech. Cheers. :^)

>>41637 Lol you don't need to apologize. I'll definitely test that in the near future

I have an project idea, but I need some general advise. The goal is to have a local LLM running on a Linux server, which it could control via CLI. The user could talk to it and give it some tasks to do, but the main part would be that when left with nothing to do, it could decide it's own goals to pursue. (coding, writing a personal website, browsing the web...) One could say that the goal of the experiment is to achieve an AI that could operate mostly by itself and achieve something meaningful while doing so. This would give the waifu the ability to have her own life outside of user interaction, making her both more useful and more realistic. I am planning on writing some LLM wrapper, which would implement API for interacting with the system, as well as handling multiple levels of memory and personality, both controlled by the model itself. More complex features like MCP API support or connection to some external hardware are being considered as possible extensions, but the current plan is a simple, Linux-controlling, AI companion. My knowledge of the current local LLM landscape is severely limited, however, so I come here to ask for help. Are local LLMs there yet? What is the least demanding model that you would consider capable of operating a Linux system and not freaking out when left alone? How powerful hardware would I need to run it? What backend should I use? (ollama, llamacpp, something else) Anything else I should know/consider?

>>41860 Hi Anon, welcome! I wish I could help you, Anon. If some other Anons here don't step up, then I'd suggest you check out /g/ 's threads on these topics in general. In particular, focus on the Local Models General ( /LMG/ ) threads. Good luck with your project, Anon! Please keep us here informed as to your progress. Cheers. :^)

>>41860 I can offer some insights. - having powerful hardware will improve quality immensely, but you can get a lower form of this up now with smaller models. - It will have context limit challenges which will need to be overcome with smart uses of tokens and how you store memories, but you can have a functioning thing as you describe, but detailed memories will eat into this heavily. having different ranked memory tiers and clearing lower ones and a mix of other clever ideas can improve all of these issues, at this stage it's just a matter of having a "lower resolution" AI companion, you could probably later update your methods as tech advances, and greater hardware means greater resolution. - I would suggest using multiple models to parse and make your AI companion smarter and not stuck into the patterns and paths that each models holds close to and deviates towards. to keep it simple, load one model, think for a bit, switch models, think a bit, especially with current limitations I would let a large chunk of time when the companion is by itself computing, to go over it's past and it's token database and refine it and have discussions and debates with itself to self-improve and refine itself. - I would have these personally as modes which you switch from her active life and talking to you, and her maintenance/sleep mode where she improves her database. - I will use sillytavern just as the idea vessel here, as I am used to it but I have not done this work myself properly since I'm busy with life. . . . use something like the memory and world lore books to store memories and events, they can each have a comprehensive recording into essentially "quants" of the lore books, where your chat history is fully recorded as non-quanted, then various levels of condensing and sorting out less important info/tokens, then your companion will think for each task and memory which quant of each lore book is suitable keeping in mind limitations hardware and current state of LLMs. - basically it will have a large database which it will parse and read through similar to how you will have it load models one at a time to use then switching to another, it will load different sets of the full database at a time to parse it. - You do the usual things you would expect, writing the personalities, creating the framework of the database, then you just have it use auto-mode similar to how sillytavern has group chats with different personality card as the other character, which is sort of her maintenance/ recursive improvement mode, again, multiple different reasoning/COT models being swapped through will be ideal here specifically. - Then you just let that be during periods of free time, and you also have her alone/ continues her life mode, which will develop memories and her character and your relationship, and to keep it coherent and immersive, I would make it follow natural time and not try to have it just create events that happened while you were gone as a task itself. The maintenance mode will be crucial here to sort and make her life events both logical and coherent as well as cutting away waste tokens, building those lore books, etc. the model changing is crucial also as even with settings and repetition penalties models slowly converge into themselves without manual maintenance that can be automated with multiple models and the mode I discussed earlier. Most of my knowledge I have is more on how you can do it abstractly, I would just vibecode and let LLMs walk you through setting it up when you get stuck. You can do all of this with smaller models and everything, but the "resolution" of your companion will be smaller, and will take longer for maintenance, and in active conversation mode I would pick your favorite model(s) and just use one as you speak to her, all of the data she needs for the conversation will be prepped as her "mental state" before from maintenance and from lore books which will fetch relevant data as you speak. The specific models I am still a newfag on, no recommendations that are any better than on this forum or halfchans /g/. Ai and LLMs run well on linux though, best of luck.

The good news here is it's possible on local, and the bulk of the work will be in maintenance mode when you are not interacting with her, but for it to be good it will be very slow, but this will be when you are sleeping or away. When you are chatting, you will simply have to accept lower quality speech to you to make it run at a reasonable tokens/second generation, but pulled from the refined database which she created during her past maintenance phase where she runs a stronger model at an abysmally slow tokens per second/minute rate for precision.

>>42608 >>42609 POTD

Well, i was thinking about this for a long time, and i came to think that its actually possible to use a LLM with an LBM at the same time with a complex mecanism in an android for less that 3k, all in one, and even making it be self-aware if you make an especial LLM. using something similar to the organization of the brain itself you can emulate the way a human being acts, talks, imagine things, have complex thoughts, its central nervous system, making it esencially think about itself as a human being. Im currently developing it by myself using something more in the way of a gladOS, just for fun and money issues, but i would like to go for a more elbaorate build. wdyt?

>>43066 Good enthusiasm, you should put that enthusiasm into learning more. LLM and LBM are already used together. LLM do not seem capable of being self aware, though that's something to hash out in the aforementioned LLM/AI thread. >Emulating the human brain There's heaps of research on this topic over the course of many years. Please read what others have achieved to better understand the problem space. >wdyt I'd work on your grammar first. Then learn more about how various AI mechanisms actually work. You have potential, please do the work and read to better understand what you're talking about, then come back and contribute.

I saw the Virtual waifu thread >>43578, and that made me think of AI chatting. I was sure there was an LLM thread, and it looks like I was correct, so I'll touch base with my current system. ST as the front end. I use Open Router for APIs >ewww corpo corporate-owned AI! I know, I'd prefer local hosting, but unless you have some A100s lying around, you won't get super far with sub 20gigs of VRAM on your GPU. For $10, you get 1000 requests a day forever for free models (This should be more than enough for the average user) >What models do I get? You get access to all the free models, and you can use the $10 of credits on the paid models. If you use paid models, that'll burn up your 10-dollars worth of credits, but you'll always get your 1k prompts per day. >What's the best free model right now? This will vary based on the bot defs you're using and the jailbreaks you utilize. I've been using Xiaomi MiMo V2 as of late, and it's pretty good. GLM 4.5 and Venice are also alright if you don't like the outputs Xiaomi is giving. >Can it do cunny? GLM 4.5? Not really, it depends on how explicit you get. Mimo V2 isn't that great. The Venice Mistral 24B is okay with it. Meta 70b is okay as well, but it tends to do a bad job of paraphrasing in most cases and will copy your descriptions word for word, which I hate. >Which is the best creative storytelling/ RP/ chat model? Depends on the defs and your prompting. Again, I start with GLM 4.5, Mimo V2, and Venice Mistral 24b. Feel free to try other free models and see how they work for you. >Do you enjoy using these API's? Your mileage will vary. I like telling stories and having bots be co-op storytellers. I'll plant a seed of a plot point and see if they're "smart enough" to figure out where I want them to go. Usually, they're good enough to get the context clues, and it's always good when it latches onto the story you're trying to tell. ERP is nice as well, but I prefer having it blended into regular RP rather than mindless goon posting. >What about chatting? I don't really do "just chatting". I should experiment more for no other reason than to spitball with specific personalities and see how good certain models work for simple chat. > What do (you) need help with? I need to learn more about jailbreaks (presets). You can edit presets, but I try not to fiddle with those too much myself. If you have any presets you'd recommend, I'd be keen to learn about them.

LLMs "General Audience" playlist by Andrej Karpathy https://www.youtube.com/playlist?list=PLAqhIrjkxbuW9U8-vZ_s_cjKPT_FqRStI

>TinyStories: How Small Can Language Models Be and Still Speak Coherent English? >abstract >Language models (LMs) are powerful tools for natural language processing, but they often struggle to produce coherent and fluent text when they are small. Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. This raises the question of whether the emergence of the ability to produce coherent English text only occurs at larger scales (with hundreds of millions of parameters or more) and complex architectures (with many layers of global attention). >In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities. >We also introduce a new paradigm for the evaluation of language models: We suggest a framework which uses GPT-4 to grade the content generated by these models as if those were stories written by students and graded by a (human) teacher. This new paradigm overcomes the flaws of standard benchmarks which often requires the model's output to be very structures, and moreover provides a multidimensional score for the model, providing scores for different capabilities such as grammar, creativity and consistency. >We hope that TinyStories can facilitate the development, analysis and research of LMs, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs. <"...especially for low-resource or specialized domains" Seems right up our alley here on /robowaifu/ . https://arxiv.org/abs/2305.07759 https://huggingface.co/datasets/roneneldan/TinyStories >related: https://github.com/karpathy/LLM101n

>"AI & ML interests >We release large pre-training datasets to accelerate open LLM development. Part of the Hugging Face Science team (hf.co/science)" https://huggingface.co/HuggingFaceFW

>>43581 Thanks for the information, Anon. Please keep us here up to date if you have further insights. Cheers. :)

>>43997 I've always felt that the next frontier (or very much should be the next frontier) of AI, is more efficient models. The current system of "bigger is better" is simply unsustainable and plateauing, not to mention the economic disruption.

>>44009 >The current system of "bigger is better" is simply unsustainable and plateauing, not to mention the economic disruption. IMHO, the most-significant issue with this current model is also the most-insidious. To wit: (((they))) have it, and you don't! :) There are other factors at play as well, but that's the fundamental issue. --- And make no mistake this is both by-design, and the evil behind it goes all the way to the top. Thankfully kikes are in their greedy, sheqel-grubbing phase just at the moment (they vacillate between only two states by and large; the other being abject fear). B/c muh_optics, this keeps them from doing absolutely everything they can to destroy the possibility of free men having private AIs, such as this kind of system (smol, local) supports.

>>44009 The current system of "bigger is better" will drive innovation into more efficient AI: there will always be folks who are driven to stick it to the man and create open source alternatives. Or to build their own cheaper alternative to monetize. IRL cyberpunks. There will always be people who want to own their software. >>43997 dafug? TinyStories sounds like it is meant to run on a potato: a raspi 5 can run qwen 1.7b via ollama blazing fast, and roleplay as a character (SPUD, obviously). If you have a gaming computer within the last decade, you can run a decent LLM :)

>>44018 >TinyStories sounds like it is meant to run on a potato That's exactly what we want, is it not, fren Mechnomancer!? :) Imagine if we could have swarms of "AIs" running on hundreds of different 'sensor/actuator' pair assemblies along all the inner & outer extremities of our robowaifus! (cf. Neuromorphics : >>12828, >>11521, et al).

>>44023 Yes, but what would be the actual hardware it would be running on? Most standard consumer computers can do basic AIs. Heck, even a LePotato (picrel) can run a small LLM. I don't think it could run on an arduino, but maybe an ESP-32? Even so, what would the point be of attaching a smol LM to a sensor? It would not "reason" in a particularly complex manner, and at that level it would probably be easier to do some brute-force coding rather than trying to 'tard wrangle an AI. Even complex AIs are a pain to 'tard wrangle, I can only imagine the troubles with a dim one. Only benefit I could see would that one that smol could be easily trained for specific tasks, but that doesn't seem to be the point of TinyStories.

>>44041 First off, let me address this point: >Even so, what would the point be of attaching a smol LM to a sensor? It's hard to convey humor through text alone, and thus why I "qualified" 'AIs'. :) I'm not actually suggesting running LLMs on smol sensor/actuator pair assemblies. >...it would probably be easier to do some brute-force coding rather than trying to 'tard wrangle an AI Yes. I'd anticipate the answer to >Yes, but what would be the actual hardware it would be running on? would be simple electronics actually, perhaps with one of the smol'st (<~$.10 cost) MCUs as the the 'brains' of the assembly. (The packaging should support SPI comms at least, however.) >tl;dr The primary point being to have hundreds of these little combo systems all cooperatively working together to run sense/actuate/sense cycles continuously across the entire robowaifu skellington system. <---> I was just using this example (initially) as an argument in favor of tiny, low-cost LLMs; then holding the idea up in favor of neuromorphics (which has a similar hardware model). This latter was simple euphemism to encapsulate the idea of hardware involved. I hope that makes sense, Mechnomancer. Thanks for pointing these issues out. Cheers. :)