s_mcleod, 8 months ago

M.2 NVMe -> PCIe x16 + some dodgy cabling = Tesla P100 eGPU 😂

Combined with my RTX3090 I can load Q4/Q5 70b models 100% into vRAM with exllama or autogpqt

#LLM #AI #ML #Llama #Nvidia #GPT

image/png
image/jpeg
image/png