加速

加速库通过仅添加四行代码，使相同的 PyTorch 代码能够在任何分布式配置上运行！简而言之，它使大规模的训练和推理变得简单、高效且适应性强。

diff

+ from accelerate import Accelerator
+ accelerator = Accelerator()

+ model, optimizer, training_dataloader, scheduler = accelerator.prepare(
+     model, optimizer, training_dataloader, scheduler
+ )

  for batch in training_dataloader:
      optimizer.zero_grad()
      inputs, targets = batch
      inputs = inputs.to(device)
      targets = targets.to(device)
      outputs = model(inputs)
      loss = loss_function(outputs, targets)
+     accelerator.backward(loss)
      optimizer.step()
      scheduler.step()

基于 torch_xla 和 torch.distributed，Accelerate 负责处理繁重的工作，因此你无需编写任何自定义代码来适应这些平台。将现有的代码库转换为利用 DeepSpeed，执行完全分片数据并行，并自动支持混合精度训练！

此代码可以通过 Accelerate 的 CLI 界面在任何系统上启动：

bash

accelerate launch {my_script.py}

Tutorials

Learn the basics and become familiar with using Accelerate. Start here if you are using Accelerate for the first time!

How-to guides

Practical guides to help you achieve a specific goal. Take a look at these guides to learn how to use Accelerate to solve real-world problems.

Conceptual guides

High-level explanations for building a better understanding of important topics such as avoiding subtle nuances and pitfalls in distributed training and DeepSpeed.

Reference

Technical descriptions of how Accelerate classes and methods work.

加速 ​

加速