https://steven-yl.github.io/mywebsite/series/pytorch%E5%AE%9E%E8%B7%B5%E6%8C%87%E5%8D%97/PyTorch实践指南 - 系列 - mywebsiteHugo -- gohugo.iozh-CNsteven@gmail.com (Steven)steven@gmail.com (Steven)This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.Fri, 03 Apr 2026 00:00:00 +0800https://steven-yl.github.io/mywebsite/batch_size_lr/Tue, 17 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/batch_size_lr/详解分布式训练中 batch size 扩大时学习率的线性缩放、平方根缩放及线性+长 warmup 的推导依据与使用建议。https://steven-yl.github.io/mywebsite/optim/Tue, 17 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/optim/解析PyTorch 优化器原理、更新公式与偏差校正推导，并给出超参数建议与优缺点总结。https://steven-yl.github.io/mywebsite/seed/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/seed/系统梳理随机种子的概念与 PyTorch 实践，包括 DataLoader worker、cuDNN、不确定性算子和常见陷阱，帮助你搭建可复现实验环境。https://steven-yl.github.io/mywebsite/lr_function/Tue, 24 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/lr_function/lr曲线图https://steven-yl.github.io/mywebsite/active_function/Tue, 24 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/active_function/本文汇总 Sigmoid、Tanh、ReLU、GELU、Swish 等激活函数，并提供分组图与总览图。https://steven-yl.github.io/mywebsite/distributed_training_guide/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/distributed_training_guide/从进程组初始化、DDP 封装、数据分片、集体通信到 Lightning 封装，全面讲解如何在单机多卡与多机多卡场景下正确使用 PyTorch 分布式训练。https://steven-yl.github.io/mywebsite/pytorch_dataset_guide/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/pytorch_dataset_guide/覆盖 map-style/IterableDataset、全部内置 Dataset 扩展、图数据与 HF datasets、典型项目扩展模式、padding 与 collate 职责划分，以及与 DataLoader 的衔接。https://steven-yl.github.io/mywebsite/net_init/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/net_init/全面对比深度学习权重初始化方法的原理、公式推导、优缺点与适用场景，附 PyTorch 代码示例和 Transformer 架构初始化最佳实践。https://steven-yl.github.io/mywebsite/dataloader_guide/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/dataloader_guide/从索引流、取样本、成 batch 三条线讲清 DataLoader 职责，涵盖 Sampler、collate_fn、num_workers、pin_memory 及与 Dataset 的衔接。https://steven-yl.github.io/mywebsite/training_solver_guide/Thu, 12 Mar 2026 00:00:00 +0800Stevenhttps://github.com/steven-ylhttps://steven-yl.github.io/mywebsite/training_solver_guide/从总览到各章节：Optimizer/SGD/Adam/AdamW 全解读、LRScheduler 族、param_groups、梯度累积与裁剪、损失选型及学习率与 batch 配置经验。