Amazon Web Services (AWS) has unveiled an open supply instrument, referred to as TorchServe, for serving PyTorch machine studying fashions. TorchServe is maintained by AWS in partnership with Facebook, which developed PyTorch, and is out there as a part of the PyTorch venture on GitHub.
Released on April 21, TorchServe is designed to make it simple to deploy PyTorch fashions at scale in manufacturing environments. Goals embody light-weight serving with low latency, and high-performance inference.
The key options of TorchServe embody:
- Default handlers for frequent functions comparable to object detection and textual content classification, sparing customers from having to jot down customized code to deploy fashions.
- Multi-model serving.
- Model versioning for A/B testing.
- Metrics for monitoring.
- RESTful endpoints for utility integration.
Any deployment surroundings might be supported by TorchServe, together with Kubernetes, Amazon SageMaker, Amazon EKS, and Amazon EC2. TorchServe requires Java 11 on Ubuntu Linux or MacOS. Detailed set up directions might be discovered on GitHub.
Copyright © 2020 IDG Communications, Inc.
