LocalAI

[MickLesk]

Name of the Script

LocalAI

Script Type

CT (LXC Container)

Does this script support arm64?

arm64 not tested

📋 Script Details

#2235

LocalAI is an OpenAI-compatible API server for local inference — text, images, transcription and TTS behind one endpoint, with a model gallery and pluggable backends. It overlaps with the existing llama.cpp script but is multi-modal rather than inference-only.

[github-actions]

The LocalAI script is ready for testing:

bash -c "$(curl -fsSL https://github.com/community-scripts/ProxmoxVED/raw/main/ct/localai.sh)"

No models are installed by default. Use the gallery in the web UI to pull one, or drop GGUF files into /opt/localai_data/models. Inference backends are fetched on first use and can be several GB each, which is what the 30 GB default disk is for.
The API is protected from the moment it starts. With user accounts off, a static API key is generated and saved to ~/localai.creds in the container; clients send it as 'Authorization: Bearer ' or an 'x-api-key' header, and it grants full admin access. With user accounts on, register in the web UI immediately after install - the first account created becomes the admin.
GPU passthrough is offered during creation and driver libraries inside the container are matched to the host driver version automatically. Without a GPU everything still works on the CPU, just slowly.
Models, backends and the auth database live in /opt/localai_data and configuration in /opt/localai.env, both outside the application directory, so updates never touch them. A mount point on /opt/localai_data is worth considering rather than growing the root disk.
User accounts need a database. Upstream ships the release binary without the CGO auth tag, so the built-in SQLite store is unavailable and PostgreSQL is installed alongside LocalAI when the option is enabled. Leaving it off keeps the container to a single service.

Note: This is not in the official repo yet—it's just a dev version! After merging into ProxmoxVE, it will need to be recreated.

Discord testing thread: https://discord.com/channels/1302816934508630047/1550399677265088562

[wittend]

Perhaps two or three problems, perhaps they are just one:
Using: var_port='8083' bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVED/main/ct/localai.sh)"
at root console on Proxmox VE.

1. Error during install:

   ---

/ / ____ _________ / / | / /
/ / / __ / / __ `/ / /| | / /
/ // // / /__/ // / / ___ |/ /
//_/_/_,/// |/___/

⚙️ Using Default Settings on node foxbat
💡 PVE Version 9.2.20 (Kernel: 7.0.14-15-pve)
🆔 Container ID: 110
🖥️ Operating System: debian (13)
📦 Container Type: Unprivileged
💾 Disk Size: 30 GB
🧠 CPU Cores: 4
🛠️ RAM Size: 8192 MiB
🎮 GPU Passthrough: Enabled
🚀 Creating a LocalAI LXC using the above default settings

✔️ Storage local (Free: 62.7GB Used: 26.5GB) [Template]
✔️ Storage zfs-vgroup (Free: 3.5TB Used: 158.8GB) [Container]
✔️ Storage 'zfs-vgroup' (zfspool) validated
✔️ Template storage 'local' validated
✔️ Template search completed
✔️ Template debian-13-standard_13.1-2_amd64.tar.zst [local]
✔️ LXC Container 110 was successfully created.
🎮 Detected Intel GPU
✔️ Automatically configuring INTEL GPU passthrough
✔️ INTEL GPU passthrough configured (2 devices)
✔️ Started LXC Container
✔️ Network in LXC is reachable (ping)
✔️ GPU passthrough configured (video:44, render:992)
✔️ Customized LXC Container
✔️ Set up Container OS
✔️ Network Connected: 10.0.0.165
✔️ IPv4 Internet Connected
✖️ IPv6 Internet Not Connected
✔️ Git DNS: github.com:(✔️ ) raw.githubusercontent.com:(✔️ ) api.github.com:(✔️ )
✔️ Updated Container OS
✔️ Installed Dependencies
✔️ Detected GPU: Intel Corporation AlderLake-S GT1 (rev 0c) (INTEL)
✔️ Deployed: libigdgmm12 (26.35.39758.10)
✔️ Deployed: intel-igc-core (2.41.5)
⠏ Fetching github release: intel-igc-opencl (2.41.5)curl: (22) The requested URL returned error: 504
💡 Download failed or stalled (attempt 1/3), retrying...
curl: (22) The requested URL returned error: 504
💡 Download failed or stalled (attempt 2/3), retrying...
curl: (22) The requested URL returned error: 504
✖️ Download failed: https://github.com/intel/intel-graphics-compiler/releases/download/v2.41.5/intel-igc-opencl-2_2.41.5%2B22716_amd64.deb
✖️ Failed to install 'intel-opencl-icd (26.35.39758.10-0)' — both apt and dpkg reported errors
→ Package intel-igc-opencl-2 is not installed.
→ dpkg: error processing package intel-opencl-icd (--install):
→ dependency problems - leaving unconfigured
→ Errors were encountered while processing:
→ intel-opencl-icd
✖️ Unmet dependencies: intel-igc-opencl-2 intel-igc-opencl-2
✖️ Hint: Run 'apt-get install -f' inside the container to attempt automatic dependency resolution.
✔️ Intel Gen 9+ GPU configured
✔️ Setup Hardware Acceleration
✔️ Deployed: localai (4.10.0)
✔️ Configured LocalAI
✔️ Created Service
✔️ Customized Container
✔️ Completed Successfully!

🚀 LocalAI setup has been successfully initialized!
💡 Access it using the following URL:
🌐 http://10.0.0.165:8080
💡 Install models from the gallery in the web UI or place GGUF files in /opt/localai_data/models

I'm not sure what to do about this.

2. Simple problem: when script ends it tells me to open <ip_addr>:8080 rather than <ip_addr>:8083 as requested.
3. When I do open the service (at 10.0.0.165:8083) I am asked for an API key. Since I chose not to install users, I don't have one or know where to get one.

My intent is to run local models on several seperate local hosts and hopefully some external ones as well (if LocalAI can do that).

Thanks for this work!

[MickLesk]

[wittend]

That seemed to do the trick.