Afternoon, regulars If anyone is interested in following along next week. All you need is a proxmox node and some ram... The project is building an inference engine on a low resource proxmox node. I will be using an N100 MiniPC. My primary use for this will be my agentic sandbox. But I also plan to send some prompts to this engine for processing in certain workflows. Fun fact: Bot Traffic overtook meatsacs in internet traffic for the first time recently. I will cover more on what all of that means next week but I would highly suggest pre-downloading the model of your choice. Each option is at least 2 GB and each one doubles up... In the PDF attachment there are links to the models you can use. Along with the resource requirements for them. You only need 1 but if you think you might be within the parameters for the next step up. You can grab two and test them. Cheers, Devon
Attachment:
models.pdf
Description: Adobe PDF document