TODO.md

AvatarStream Refactor TODOs

Phase 1: Environment & Setup

• Create TODO.md
• Dependency Management:
  • Analyze Python imports in game/AvatarStream/scripts/python/holistic_tracker.py
  • Update game/AvatarStream/scripts/python/requirements.txt (include opencv-python, mediapipe, pyvirtualcam)
• Unified Launcher (run.py):
  • Create run.py at the root
  • Detect OS (Windows/Linux/Mac)
  • Verify dependencies are installed
  • Launch Python tracking script in background
  • Launch Godot executable (configurable via .env or args)
  • Gracefully kill processes on exit

Phase 2: Cross-Platform Virtual Camera

• Modify Godot Project:
  • Create script to capture Viewport texture
  • Send frame data back to Python (UDP/TCP)
• Update Python Script:
  • Integrate pyvirtualcam
  • Add listener thread for Godot frames
  • Push frames to virtual camera

Phase 3: Mobile & Cleanup

• Mobile Logic:
  • Add OS.get_name() check in Godot
  • Disable frame-sending logic on Android/iOS
• Documentation:
  • Update README.md with run.py instructions
  • Add setup notes for Windows (OBS) and Linux (v4l2loopback)

Phase 4: Optimization & UX (Current)

• Performance:
  • Switch Pose Data from JSON to Binary (UDP) in holistic_tracker.py and MediaPipeBridge.gd (~80% size reduction)
  • Switch Video Stream from MJPEG to Raw RGB (TCP) in VirtualCameraSender.gd and holistic_tracker.py (Eliminate encode/decode CPU cost)
• Setup & UX:
  • Auto-install Python dependencies in run.py if missing
  • Fix double-launch bug: Godot no longer launches Python if run.py already did (via AVATARSTREAM_LAUNCHED_BY_RUNNER env var)
• Quality:
  • Add protocol unit tests (game/AvatarStream/scripts/python/tests/test_protocol.py)