• Hacker News
  • new|
  • comments|
  • show|
  • ask|
  • jobs|
  • PcChip 37 minutes

    I didn't see any benchmarks against vllm, sglang, exllama, etc

    rancor 29 minutes

    Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.