I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.
The bug
Description:
Summary
- The current RKNN pool implementation in machine-learning/immich_ml/sessions/rknn/rknnpool.py uses a shared queue and separate put() and get() methods. When multiple threads call RknnSession.run concurrently, put()/get() pairs can interleave and a caller may receive another thread's result. This is a race condition and can cause incorrect inference results.
- There is also a race condition on
self.num += 1 which may cause the jobs distributed unevenly on the three NPU cores.
The OS that Immich Server is running on
Ubuntu 24.04
Version of Immich Server
v3.1.0
Version of Immich Mobile App
v3.1.0
Platform with the issue
Device make and model
Orange Pi 5B
Your docker-compose.yml content
name: immich
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
extends:
file: hwaccel.transcoding.yml
service: rkmpp # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
ports:
- 2283:2283
depends_on:
- redis
- database
restart: always
healthcheck:
disable: false
immich-machine-learning:
container_name: immich_machine_learning
# For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
# Example tag: ${IMMICH_VERSION:-release}-cuda
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-rknn
extends: # uncomment this section for hardware acceleration - see https://docs.immich.app/features/ml-hardware-acceleration
file: hwaccel.ml.yml
service: rknn # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
volumes:
- model-cache:/cache
- /dev/dri:/dev/dri
env_file:
- .env
environment:
HF_TOKEN: ****
MACHINE_LEARNING_RKNN_THREADS: 3
restart: always
healthcheck:
disable: false
redis:
container_name: immich_redis
image: docker.io/valkey/valkey:8-bookworm@sha256:fea8b3e67b15729d4bb70589eb03367bab9ad1ee89c876f54327fc7c6e618571
healthcheck:
test: redis-cli ping || exit 1
restart: always
database:
container_name: immich_postgres
image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0
environment:
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_USER: ${DB_USERNAME}
POSTGRES_DB: ${DB_DATABASE_NAME}
POSTGRES_INITDB_ARGS: '--data-checksums'
volumes:
- ${DB_DATA_LOCATION}:/var/lib/postgresql/data
shm_size: 128mb
restart: always
volumes:
model-cache:Your .env content
# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables
# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=./postgres
# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Europe/Amsterdam
# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v3
# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=******
Reproduction steps
- Concurrent machine learning jobs trigger two threads (A and B).
- Thread A calls RknnSession.run(...) which internally calls rknnpool.put(...) then is preempted.
- Thread B calls RknnSession.run(...) (put + get) and its get() consumes the output produced for A.
- Thread A later resumes and calls get() and receives B's output (or blocks if queue empty).
Relevant log output
Additional information
No response
I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.
The bug
Description:
Summary
self.num += 1which may cause the jobs distributed unevenly on the three NPU cores.The OS that Immich Server is running on
Ubuntu 24.04
Version of Immich Server
v3.1.0
Version of Immich Mobile App
v3.1.0
Platform with the issue
Device make and model
Orange Pi 5B
Your docker-compose.yml content
Your .env content
Reproduction steps
Relevant log output
Additional information
No response