TensorRT LLM - NVIDIA's open-source large model inference optimization framework
TensorRT LLM is an inference performance framework for large language models (LLMs) on NVIDIA GPUs, developed by NVIDIA. Based on the PyTorch architecture, TensorRT LLM provides an efficient and easy-to-use Python API...