SlavaVlad 16bdf2bd00 Fix CUDA/ROCm compatibility: default to CPU for AMD GPUs ...

- Change default device to CPU in docker-compose and main.py
- Set compute_type=int8 for CPU inference
- Auto-detect device if env not set

2026-05-13 03:50:58 +03:00

..

__init__.py

Add OpenAI-compatible API and Docker deployment

2026-05-13 01:37:47 +03:00

main.py

Fix CUDA/ROCm compatibility: default to CPU for AMD GPUs

2026-05-13 03:50:58 +03:00

serve.py

Add OpenAI-compatible API and Docker deployment

2026-05-13 01:37:47 +03:00