急急急，当数据路径存在多个文件时，会报错显示 no sample to consume: 0

一、问题现象（附报错日志上下文）：
如果data path存在多个文件，或者写明权重，就会出现报错。（实测单独使用任意一个数据集文件都能正常运行）
单个数据集文件，声明权重，报相同的错。
多个数据集文件，无论是否声明权重，都会报错，且错误相同。
任意单个数据集文件，不声明权重，可以正常运行。

`  data_path ....................................... ['1', '/home/ma-user/work/dnadata/gene_HG00103_text_document', '1', '/home/ma-user/work/dnadata/gene_HG00113_text_document']`
二、软件版本:
Megatron版本：0.8.0
MindSpeed版本：0.8.0
CANN套件版本：8.0rc3 已安装NNCL并设置环境变量

```
> finished creating GPT datasets ...
Traceback (most recent call last):
  File "/home/ma-user/CusEnv/Megatron-LM/pretrain_gpt.py", line 246, in <module>
    pretrain(
  File "/home/ma-user/CusEnv/MindSpeed/mindspeed/core/training.py", line 512, in wrapper
    pretrain(*args, **kwargs)
  File "/home/ma-user/CusEnv/MindSpeed/mindspeed/training.py", line 251, in pretrain
    = build_train_valid_test_data_iterators(
  File "/home/ma-user/CusEnv/Megatron-LM/megatron/training/training.py", line 1515, in build_train_valid_test_data_iterators
    build_train_valid_test_data_loaders(
  File "/home/ma-user/CusEnv/Megatron-LM/megatron/training/training.py", line 1484, in build_train_valid_test_data_loaders
    valid_dataloader = build_pretraining_data_loader(
  File "/home/ma-user/CusEnv/MindSpeed/mindspeed/core/performance/auto_pipeline_perf/data_samplers.py", line 28, in wrapper
    dataloader = build_pretraining_data_loader(*args, **kwargs)
  File "/home/ma-user/CusEnv/Megatron-LM/megatron/legacy/data/data_samplers.py", line 23, in build_pretraining_data_loader
    batch_sampler = MegatronPretrainingSampler(
  File "/home/ma-user/CusEnv/Megatron-LM/megatron/legacy/data/data_samplers.py", line 68, in __init__
    assert self.total_samples > 0, \
AssertionError: no sample to consume: 0
```

Ascend/MindSpeed

内容风险标识

评论 (2)

Ascend/MindSpeed .gitee-modal { width: 500px !important; }

内容风险标识