测试时:megengine._internal.exc.MegBrainError: MegBrain core throws exception: mgb::cg::GraphError

为使您的问题得到快速解决,建议参考以下模板:

【标题】
(简洁、精准的描述您的问题,例如“int8模型,多次抽feature,存在可见误差”)
【版本、环境信息】

  • MegEngine 版本:0.3.
  • GPU型号:3090
  • 系统环境:16.04
  • python版本: 3.6.9

【如为报错请提供以下复现信息】
0%| | 0/4370 [00:00<?, ?it/s]03 17:05:42[mgb] WRN indexing_multi_axis_vec(concat[4117])[4131]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:42[mgb] WRN indexing_multi_axis_vec(concat[4117])[4150]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:42[mgb] WRN indexing_multi_axis_vec(concat[4117])[4169]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
0%| | 1/4370 [26:25<1924:12:08, 1585.52s/it]03 17:05:43[mgb] WRN indexing_multi_axis_vec(concat[24370])[24520]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:43[mgb] WRN indexing_multi_axis_vec(concat[24370])[24513]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:43[mgb] WRN indexing_multi_axis_vec(concat[24370])[24506]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:43[mgb] WRN indexing_multi_axis_vec(concat[24370])[24499]{IndexingMultiAxisVec}: no vector indexer; consider using Subtensor family for better performance; you can set MGB_THROW_ON_SCALAR_IDX to throw an exception to help tracking the related operator
03 17:05:43[mgb] ERR error occurred in computing sequence; synchronizing all comp nodes and releasing vars now …
Process Process-1:
Traceback (most recent call last):
File “/home/a230/.conda/envs/megengine/lib/python3.6/multiprocessing/process.py”, line 258, in _bootstrap
self.run()
File “/home/a230/.conda/envs/megengine/lib/python3.6/multiprocessing/process.py”, line 93, in run
self._target(*self._args, **self._kwargs)
File “test.py”, line 69, in inference
pred_boxes = val_func().numpy()
File “/home/a230/.conda/envs/megengine/lib/python3.6/site-packages/megengine/jit/init.py”, line 386, in call
self._compiled_func()
File “/home/a230/.conda/envs/megengine/lib/python3.6/site-packages/megengine/_internal/mgb.py”, line 1109, in call
self._execute()
File “/home/a230/.conda/envs/megengine/lib/python3.6/site-packages/megengine/_internal/mgb.py”, line 993, in _execute
return _mgb.AsyncExec__execute(self)
megengine._internal.exc.MegBrainError: MegBrain core throws exception: mgb::cg::GraphError
var {id:24530, shape:{0,256,7,7}, Float32, owner:indexing_multi_axis_vec(concat[24527])[24529]{IndexingMultiAxisVec}, name:indexing_multi_axis_vec(concat[24527])[24529], slot:0, gpu0:0, d, 8, 1} has empty memplan, but allowed=1 receiver=VarReceiverInfo(nr_direct_comp_req=0 dev_value=1, host_value=0, shape=0, allow_empty_value=0)

  • bt:/home/a230/.conda/envs/megengine/lib/python3.6/site-packages/megengine/_internal/_mgb.cpython-36m-x86_64-linux-gnu.so{1c6249d,1c65bc2,1c1c191,1c1b777}
    | Associated operator: id=24529 name=indexing_multi_axis_vec(concat[24527])[24529] type=mgb::opr::IndexingMultiAxisVec
    | input variables:
    | 0: {id:24528, shape:{4,256,7,7}, Float32, owner:concat(roi_align[24502]:o0,roi_align[24509]:o0,roi_align[24516]:o0,…)[24527]{Concat}, name:concat(roi_align[24502]:o0,roi_align[24509]:o0,roi_align[24516]:o0,…)[24527], slot:0, gpu0:0, d, 8, 8}
    | 1: {id:24464, shape:{0}, Int32, owner:cond_take(indexing_multi_axis_vec[24459],indexing_multi_axis_vec[24459])[24462]{CondTake}, name:cond_take(indexing_multi_axis_vec[24459],indexing_multi_axis_vec[24459])[24462]:o1, slot:1, gpu0:0, d, 1, 1}
    | output variables:
    | 0: {id:24530, shape:{0,256,7,7}, Float32, owner:indexing_multi_axis_vec(concat[24527])[24529]{IndexingMultiAxisVec}, name:indexing_multi_axis_vec(concat[24527])[24529], slot:0, gpu0:0, d, 8, 1}
    | 1: {id:24531, shape:{0}, Byte, owner:indexing_multi_axis_vec(concat[24527])[24529]{IndexingMultiAxisVec}, name:indexing_multi_axis_vec(concat[24527])[24529]:workspace, slot:1, gpu0:0, d, 8, 1}
    |
    | Unoptimized equivalent of associated operator: id=4208 name=indexing_multi_axis_vec(concat[4197])[4208] type=mgb::opr::IndexingMultiAxisVec
    | input variables:
    | 0: {id:4198, shape:{1004,256,7,7}, Float32, owner:concat(roi_align[4134]:o0,roi_align[4153]:o0,roi_align[4172]:o0,…)[4197]{Concat}, name:concat(roi_align[4134]:o0,roi_align[4153]:o0,roi_align[4172]:o0,…)[4197], slot:0, gpu0:0, d, 8, 8}
    | 1: {id:4207, shape:{1000}, Int32, owner:set_grad(cond_take[4202]:o1)[4206]{SetGrad}, name:set_grad(cond_take[4202]:o1)[4206], slot:0, gpu0:0, d, 8, 8}
    | output variables:
    | 0: {id:4209, shape:{1000,256,7,7}, Float32, owner:indexing_multi_axis_vec(concat[4197])[4208]{IndexingMultiAxisVec}, name:indexing_multi_axis_vec(concat[4197])[4208], slot:0, gpu0:0, d, 8, 1}
    | 1: {id:4210, shape:{4000}, Byte, owner:indexing_multi_axis_vec(concat[4197])[4208]{IndexingMultiAxisVec}, name:indexing_multi_axis_vec(concat[4197])[4208]:workspace, slot:1, gpu0:0, d, 8, 1}

可以提供一下test.py的代码吗?

cond_take取出来是个空的tensor,但是后面接的IndexingMultiAxisVec op不允许输出为空tensor,也许可以调一下cond_take的阈值,让选出来的tensor不为空。

@o15124046923 建议你用最新的版本,0.3.1 的版本中遗留的历史问题比较多