InferCap
A toolkit for checking whether an LLM can run on available hardware, generating a vLLM serving configuration, and measuring inference capacity.
The problem
Deploying an LLM often starts with uncertainty: will the model fit, which serving settings are reasonable, and where does the GPU reach its limit?
The approach
InferCap combines environment and GPU checks, model-family discovery, static weight-memory estimates, endpoint verification, benchmark modes, and telemetry into one CLI workflow.
Available materials
The public repository includes a Python CLI, tests, docs, benchmark JSON / PNG outputs, and a live web interface for exploring the tool.
Contribution & context
Designed and built the project as an open-source toolkit.
Project scope & limitations
The current implementation targets NVIDIA GPUs and vLLM. Memory estimates are feasibility checks and do not guarantee runtime stability.