When you send a prompt to an LLM, three layers shape how fast you get a response: the hardware (GPUs, TPUs, LPUs), the model size and architecture, and the inference engine strategies sitting on top. Most of the latency battle is fought at that third layer — and the core problem …