Abstract:
This paper outlines how we modified the torch2trt library which allowed us to build a recursive framework that can quantize previously unsupported PyTorch models. The fra...Show MoreMetadata
Abstract:
This paper outlines how we modified the torch2trt library which allowed us to build a recursive framework that can quantize previously unsupported PyTorch models. The framework partitions the PyTorch model into supported and unsupported modules, and then rebuilds the PyTorch model by replacing the supported PyTorch modules with faster TensorRT modules. The framework allows us to optimize and deploy more advanced Deep Neural Network algorithms that are not natively supported by torch2trt.
Published in: 2023 IEEE International Conference on Big Data (BigData)
Date of Conference: 15-18 December 2023
Date Added to IEEE Xplore: 22 January 2024
ISBN Information: