- en: TorchServe
  id: totrans-0
  prefs:
  - PREF_H1
  type: TYPE_NORMAL
  zh: TorchServe
- en: 原文：[https://pytorch.org/serve](https://pytorch.org/serve)
  id: totrans-1
  prefs:
  - PREF_BQ
  type: TYPE_NORMAL
  zh: 原文：[https://pytorch.org/serve](https://pytorch.org/serve)
- en: TorchServe is a performant, flexible and easy to use tool for serving PyTorch
    models in production.
  id: totrans-2
  prefs: []
  type: TYPE_NORMAL
  zh: TorchServe是一个性能优越、灵活且易于使用的工具，用于在生产环境中提供PyTorch模型。
- en: What’s going on in TorchServe?
  id: totrans-3
  prefs: []
  type: TYPE_NORMAL
  zh: TorchServe中发生了什么？
- en: '[High performance Llama 2 deployments with AWS Inferentia2 using TorchServe](https://pytorch.org/blog/high-performance-llama/)'
  id: totrans-4
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用AWS Inferentia2和TorchServe进行高性能Llama 2部署](https://pytorch.org/blog/high-performance-llama/)'
- en: '[Naver Case Study: Transition From High-Cost GPUs to Intel CPUs and oneAPI
    powered Software with performance](https://pytorch.org/blog/ml-model-server-resource-saving/)'
  id: totrans-5
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[Naver案例研究：从高成本GPU过渡到Intel CPU和使用性能的oneAPI软件](https://pytorch.org/blog/ml-model-server-resource-saving/)'
- en: '[Run multiple generative AI models on GPU using Amazon SageMaker multi-model
    endpoints with TorchServe and save up to 75% in inference costs](https://aws.amazon.com/blogs/machine-learning/run-multiple-generative-ai-models-on-gpu-using-amazon-sagemaker-multi-model-endpoints-with-torchserve-and-save-up-to-75-in-inference-costs/)'
  id: totrans-6
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用Amazon SageMaker多模型端点在GPU上运行多个生成AI模型，并节省高达75%的推理成本](https://aws.amazon.com/blogs/machine-learning/run-multiple-generative-ai-models-on-gpu-using-amazon-sagemaker-multi-model-endpoints-with-torchserve-and-save-up-to-75-in-inference-costs/)'
- en: '[Deploying your Generative AI model in only four steps with Vertex AI and PyTorch](https://cloud.google.com/blog/products/ai-machine-learning/get-your-genai-model-going-in-four-easy-steps)'
  id: totrans-7
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用Vertex AI和PyTorch在四个步骤中部署您的生成AI模型](https://cloud.google.com/blog/products/ai-machine-learning/get-your-genai-model-going-in-four-easy-steps)'
- en: '[PyTorch Model Serving on Google Cloud TPUv5](https://cloud.google.com/tpu/docs/v5e-inference#pytorch-model-inference-and-serving)'
  id: totrans-8
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[在Google Cloud TPUv5上提供PyTorch模型](https://cloud.google.com/tpu/docs/v5e-inference#pytorch-model-inference-and-serving)'
- en: '[Monitoring using Datadog](https://www.datadoghq.com/blog/ai-integrations/#model-serving-and-deployment-vertex-ai-amazon-sagemaker-torchserve)'
  id: totrans-9
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用Datadog进行监控](https://www.datadoghq.com/blog/ai-integrations/#model-serving-and-deployment-vertex-ai-amazon-sagemaker-torchserve)'
- en: '[Torchserve Performance Tuning, Animated Drawings Case-Study](https://pytorch.org/blog/torchserve-performance-tuning/)'
  id: totrans-10
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[Torchserve性能调优，动画绘图案例研究](https://pytorch.org/blog/torchserve-performance-tuning/)'
- en: '[Walmart Search: Serving Models at a Scale on TorchServe](https://medium.com/walmartglobaltech/search-model-serving-using-pytorch-and-torchserve-6caf9d1c5f4d)'
  id: totrans-11
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[Walmart搜索：在TorchServe上规模化提供模型](https://medium.com/walmartglobaltech/search-model-serving-using-pytorch-and-torchserve-6caf9d1c5f4d)'
- en: '[Scaling inference on CPU with TorchServe](https://www.youtube.com/watch?v=066_Jd6cwZg)'
  id: totrans-12
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用TorchServe在CPU上扩展推理](https://www.youtube.com/watch?v=066_Jd6cwZg)'
- en: '[TorchServe C++ backend](https://www.youtube.com/watch?v=OSmGGDpaesc)'
  id: totrans-13
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[TorchServe C++后端](https://www.youtube.com/watch?v=OSmGGDpaesc)'
- en: '[Grokking Intel CPU PyTorch performance from first principles: a TorchServe
    case study](https://pytorch.org/tutorials/intermediate/torchserve_with_ipex.html)'
  id: totrans-14
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[从第一原理理解英特尔CPU PyTorch性能：TorchServe案例研究](https://pytorch.org/tutorials/intermediate/torchserve_with_ipex.html)'
- en: '[Grokking Intel CPU PyTorch performance from first principles( Part 2): a TorchServe
    case study](https://pytorch.org/tutorials/intermediate/torchserve_with_ipex_2.html)'
  id: totrans-15
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[从第一原理理解英特尔CPU PyTorch性能（第2部分）：TorchServe案例研究](https://pytorch.org/tutorials/intermediate/torchserve_with_ipex_2.html)'
- en: '[Case Study: Amazon Ads Uses PyTorch and AWS Inferentia to Scale Models for
    Ads Processing](https://pytorch.org/blog/amazon-ads-case-study/)'
  id: totrans-16
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[案例研究：亚马逊广告使用PyTorch和AWS Inferentia扩展广告处理模型](https://pytorch.org/blog/amazon-ads-case-study/)'
- en: '[Optimize your inference jobs using dynamic batch inference with TorchServe
    on Amazon SageMaker](https://aws.amazon.com/blogs/machine-learning/optimize-your-inference-jobs-using-dynamic-batch-inference-with-torchserve-on-amazon-sagemaker/)'
  id: totrans-17
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用TorchServe在Amazon SageMaker上进行动态批量推理来优化推理作业](https://aws.amazon.com/blogs/machine-learning/optimize-your-inference-jobs-using-dynamic-batch-inference-with-torchserve-on-amazon-sagemaker/)'
- en: '[Using AI to bring children’s drawings to life](https://ai.facebook.com/blog/using-ai-to-bring-childrens-drawings-to-life/)'
  id: totrans-18
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[使用AI让儿童的绘画栩栩如生](https://ai.facebook.com/blog/using-ai-to-bring-childrens-drawings-to-life/)'
- en: '[Model Serving in PyTorch](https://www.youtube.com/watch?v=2A17ZtycsPw)'
  id: totrans-19
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[在PyTorch中提供模型](https://www.youtube.com/watch?v=2A17ZtycsPw)'
- en: '[Evolution of Cresta’s machine learning architecture: Migration to AWS and
    PyTorch](https://aws.amazon.com/blogs/machine-learning/evolution-of-crestas-machine-learning-architecture-migration-to-aws-and-pytorch/)'
  id: totrans-20
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[克雷斯塔机器学习架构的演变：迁移到AWS和PyTorch](https://aws.amazon.com/blogs/machine-learning/evolution-of-crestas-machine-learning-architecture-migration-to-aws-and-pytorch/)'
- en: '[Explain Like I’m 5: TorchServe](https://www.youtube.com/watch?v=NEdZbkfHQCk)'
  id: totrans-21
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[像我5岁一样解释：TorchServe](https://www.youtube.com/watch?v=NEdZbkfHQCk)'
- en: '[How to Serve PyTorch Models with TorchServe](https://www.youtube.com/watch?v=XlO7iQMV3Ik)'
  id: totrans-22
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[如何使用TorchServe提供PyTorch模型](https://www.youtube.com/watch?v=XlO7iQMV3Ik)'
- en: '[How to deploy PyTorch models on Vertex AI](https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai)'
  id: totrans-23
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[如何在Vertex AI上部署PyTorch模型](https://cloud.google.com/blog/topics/developers-practitioners/pytorch-google-cloud-how-deploy-pytorch-models-vertex-ai)'
- en: '[Quantitative Comparison of Serving Platforms](https://biano-ai.github.io/research/2021/08/16/quantitative-comparison-of-serving-platforms-for-neural-networks.html)'
  id: totrans-24
  prefs:
  - PREF_UL
  type: TYPE_NORMAL
  zh: '[服务平台的定量比较](https://biano-ai.github.io/research/2021/08/16/quantitative-comparison-of-serving-platforms-for-neural-networks.html)'
- en: All
  id: totrans-25
  prefs: []
  type: TYPE_NORMAL
  zh: 全部
- en: '* * *'
  id: totrans-26
  prefs: []
  type: TYPE_NORMAL
  zh: '* * *'
- en: '[#### TorchServe Quick Start'
  id: totrans-27
  prefs: []
  type: TYPE_NORMAL
  zh: '[#### TorchServe快速入门'
- en: 'Topics: Quick Start'
  id: totrans-28
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：快速入门
- en: Learn how to install TorchServe and serve models.
  id: totrans-29
  prefs: []
  type: TYPE_NORMAL
  zh: 学习如何安装TorchServe并提供模型。
- en: '![](../Images/2e44a4dab4c1bd5cde13eaa681343e78.png)](getting_started.html)
    [#### Running TorchServe'
  id: totrans-30
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/2e44a4dab4c1bd5cde13eaa681343e78.png)](getting_started.html)
    [#### 运行TorchServe'
- en: 'Topics: Running TorchServe'
  id: totrans-31
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：运行TorchServe
- en: Indepth explanation of how to run TorchServe
  id: totrans-32
  prefs: []
  type: TYPE_NORMAL
  zh: 深入解释如何运行TorchServe
- en: '![](../Images/661e92286b91a04a664aa0dd434223f4.png)](server.html) [#### Why
    TorchServe'
  id: totrans-33
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/661e92286b91a04a664aa0dd434223f4.png)](server.html) [#### 为什么选择TorchServe'
- en: 'Topics: Examples'
  id: totrans-34
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：示例
- en: Various TorchServe use cases
  id: totrans-35
  prefs: []
  type: TYPE_NORMAL
  zh: 各种TorchServe用例
- en: '![](../Images/0507eb3112fdbfd24e3e2ba13aa3e3fa.png)](use_cases.html) [####
    Performance'
  id: totrans-36
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/0507eb3112fdbfd24e3e2ba13aa3e3fa.png)](use_cases.html) [####
    性能'
- en: 'Topics: Performance,Troubleshooting'
  id: totrans-37
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：性能，故障排除
- en: Guides and best practices on how to improve perfromance when working with TorchServe
  id: totrans-38
  prefs: []
  type: TYPE_NORMAL
  zh: 指南和最佳实践，以提高在使用TorchServe时的性能
- en: '![](../Images/a115bf3860d7637d64025cdabc4de95b.png)](performance_guide.html)
    [#### Metrics'
  id: totrans-39
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/a115bf3860d7637d64025cdabc4de95b.png)](performance_guide.html)
    [#### 指标'
- en: 'Topics: Metrics,Performance,Troubleshooting'
  id: totrans-40
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：指标，性能，故障排除
- en: Collecting and viewing Torcherve metrics
  id: totrans-41
  prefs: []
  type: TYPE_NORMAL
  zh: 收集和查看Torcherve指标
- en: '![](../Images/eab661f8c4941205ffdc566aced9bccf.png)](metrics.html) [#### Large
    Model Inference'
  id: totrans-42
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/eab661f8c4941205ffdc566aced9bccf.png)](metrics.html) [#### 大型模型推理'
- en: 'Topics: Large-Models,Performance'
  id: totrans-43
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：大型模型，性能
- en: Serving Large Models with TorchServe
  id: totrans-44
  prefs: []
  type: TYPE_NORMAL
  zh: 使用TorchServe为大型模型提供服务
- en: '![](../Images/f6afe69d86ffcf863cd832ed3698732f.png)](large_model_inference.html)
    [#### Troubleshooting'
  id: totrans-45
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/f6afe69d86ffcf863cd832ed3698732f.png)](large_model_inference.html)
    [#### 故障排除'
- en: 'Topics: Troubleshooting,Performance'
  id: totrans-46
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：故障排除，性能
- en: Various updates on Torcherve and use cases.
  id: totrans-47
  prefs: []
  type: TYPE_NORMAL
  zh: 有关Torcherve和用例的各种更新。
- en: '![](../Images/d23903f23b5705cc9f1d9bdca6ce6bbb.png)](Troubleshooting.html)
    [#### TorchServe Security Policy'
  id: totrans-48
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/d23903f23b5705cc9f1d9bdca6ce6bbb.png)](Troubleshooting.html)
    [#### TorchServe安全策略'
- en: 'Topics: Security'
  id: totrans-49
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：安全
- en: Security Policy
  id: totrans-50
  prefs: []
  type: TYPE_NORMAL
  zh: 安全策略
- en: '![](../Images/2e44a4dab4c1bd5cde13eaa681343e78.png)](security.html) [#### FAQs'
  id: totrans-51
  prefs: []
  type: TYPE_NORMAL
  zh: '![](../Images/2e44a4dab4c1bd5cde13eaa681343e78.png)](security.html) [#### 常见问题解答'
- en: 'Topics: FAQS'
  id: totrans-52
  prefs: []
  type: TYPE_NORMAL
  zh: 主题：常见问题解答
- en: Various frequently asked questions.
  id: totrans-53
  prefs: []
  type: TYPE_NORMAL
  zh: 各种常见问题。
- en: '![](../Images/7ccfac0b40fe2fac42582244489f0da4.png)](FAQs.html)'
  id: totrans-54
  prefs: []
  type: TYPE_IMG
  zh: '![](../Images/7ccfac0b40fe2fac42582244489f0da4.png)](FAQs.html)'