为使您的问题得到快速解决，建议参考以下模板：
【标题】
（简洁、精准的描述您的问题，例如“int8模型，多次抽feature，存在可见误差”）
【版本、环境信息】
【模型信息】
【Load_and_run LOG】
【如为报错请提供以下复现信息】
【标题】
执行课堂作业（python数据处理实践）报错
【版本、环境信息】
WS2环境
python：3.6.9
【源码】
import cv2
import nori2 as nori
from meghair.utils import io
from meghair.utils.imgproc import imdecode
fetcher = nori.Fetcher()
datas = io.load(“s3://ai-cultivate/task1/pics.pkl”)
for item in datas:
print(item)
img = imdecode(fetcher.get(item[“nori_id”]))[…, :3]
cv2.imwrite("%s.jpg" % item[“meta”], img)
break
【执行命令】
python3 test_nori.py
【报错信息】
`wangcong@pensive-agnesi:/data/wangcong_test$ rlaunch --cpu=1 --gpu=1 --memory=4096 – python3 test_nori.py
/usr/lib/python3/dist-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn’t match a supported version!
RequestsDependencyWarning)
18 15:01:02 MegSkull 4.10.0 initialized. Using MegBrain: 8.10.1
18 15:01:02 MegHair 4.7.0 initialized
failed to process ‘HeadObject’, with parameters: {‘url_path’: ‘/ch-share/pics.pkl’, ‘query_string’: {}, ‘method’: ‘HEAD’, ‘headers’: {‘User-Agent’: ‘Boto3/1.17.73 Python/3.6.9 Linux/4.15.0-136-generic Botocore/1.20.73’}, ‘body’: b’’, ‘url’: ‘http://oss.i.brainpp.cn/ch-share/pics.pkl’, ‘context’: {‘client_region’: ‘us-east-1’, ‘client_config’: <botocore.config.Config object at 0x7f20645eeeb8>, ‘has_streaming_input’: False, ‘auth_type’: None, ‘signing’: {‘bucket’: ‘ch-share’}}}
Traceback (most recent call last):
File “/usr/local/lib/python3.6/dist-packages/refile/s3.py”, line 301, in s3_isfile
client.head_object(Bucket=bucket, Key=key)
File “/usr/local/lib/python3.6/dist-packages/botocore/client.py”, line 386, in _api_call
return self._make_api_call(operation_name, kwargs)
File “/usr/local/lib/python3.6/dist-packages/botocore/client.py”, line 692, in _make_api_call
operation_model, request_dict, request_context)
File “/usr/local/lib/python3.6/dist-packages/refile/errors.py”, line 103, in wrapper
result = func(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/botocore/client.py”, line 711, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File “/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py”, line 102, in make_request
return self._send_request(request_dict, operation_model)
File “/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py”, line 132, in _send_request
request = self.create_request(request_dict, operation_model)
File “/usr/local/lib/python3.6/dist-packages/botocore/endpoint.py”, line 116, in create_request
operation_name=operation_model.name)
File “/usr/local/lib/python3.6/dist-packages/botocore/hooks.py”, line 356, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/botocore/hooks.py”, line 228, in emit
return self._emit(event_name, kwargs)
File “/usr/local/lib/python3.6/dist-packages/botocore/hooks.py”, line 211, in _emit
response = handler(**kwargs)
File “/usr/local/lib/python3.6/dist-packages/botocore/signers.py”, line 90, in handler
return self.sign(operation_name, request)
File “/usr/local/lib/python3.6/dist-packages/botocore/signers.py”, line 162, in sign
auth.add_auth(request)
File “/usr/local/lib/python3.6/dist-packages/botocore/auth.py”, line 373, in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “test_nori.py”, line 9, in
datas = io.load(“s3://ch-share/pics.pkl”)
File “/usr/local/lib/python3.6/dist-packages/meghair/utils/io.py”, line 395, in load
with fopen(fname, ‘rb’) as fin:
File “/usr/local/lib/python3.6/dist-packages/refile/smart.py”, line 387, in smart_open
return SmartPath(path).open(mode, **options)
File “/usr/local/lib/python3.6/dist-packages/refile/smart_path.py”, line 85, in open
return self.path.open(mode, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/refile/s3_path.py”, line 98, in open
if not self.is_file():
File “/usr/local/lib/python3.6/dist-packages/refile/s3_path.py”, line 36, in is_file
return s3_isfile(self.path_with_protocol)
File “/usr/local/lib/python3.6/dist-packages/refile/s3.py”, line 305, in s3_isfile
raise error
refile.errors.S3ConfigError: Unable to locate credentials
INFO[0014] [Agent: A18890146806741] Worker FAILED: user command error: exit code 1
wangcong@pensive-agnesi:/data/wangcong_test$ `
你看下你是否有s3的权限，先oss ls看下
提示oss command not found，我先配置下再试试
wangcong@pensive-agnesi:/data/wangcong_test1$ oss ls
2021-04-08 22:02:47 ai-learning-team-one
2020-07-28 10:34:59 ais-poc-asset
2020-07-17 16:20:58 algo-mass-production
2020-02-10 16:55:47 arxiv-data
2020-04-23 11:10:47 awda
2020-04-23 11:10:56 awdaawd
2020-04-30 13:47:37 awesome-neupeak
2021-04-15 14:21:46 basecls
2020-02-27 11:05:18 basedet-pub
2020-05-18 02:54:18 bba-experience-test
2020-11-10 20:11:52 bppcore-test
2020-02-26 17:34:34 brain-ftp
2020-02-10 17:10:59 brainpp-py-sdk
2020-04-14 19:59:45 bug-report
2020-02-10 17:11:00 bypshared
2019-09-19 17:17:45 caobin-data
2020-02-10 17:11:01 ceph-base-model-zoo
2021-02-07 16:02:22 chaiyizhe-data-processing-oss
2020-02-10 17:25:18 challenge-dataset
2021-04-15 17:06:32 chenkexin-data
2020-06-10 19:13:06 chenxiyang
2020-08-10 15:04:45 cluster-data
2020-08-17 16:49:33 cluster-site
2021-02-05 14:41:55 cnngeometric
2020-12-08 17:19:31 collect-visible-data
2020-03-27 16:03:55 crawler
2019-12-18 17:41:19 cvpack
2020-05-16 18:58:37 cvpack2dumps
2020-05-21 10:26:42 cyz-work
2020-02-12 17:44:32 data-for-activity
2020-01-03 18:37:01 det-terminal-share
2021-01-04 17:18:28 distort-db
2019-11-09 09:04:16 djh-model-share
2020-03-24 11:20:37 djh-model-share-wha
2021-04-06 21:06:36 dogran-share
2020-02-12 16:59:40 dongruixiao-data-processing-oss
2021-02-23 13:32:47 douwenhao-data-processing-oss
2019-12-05 15:26:39 drx
2019-11-06 11:57:19 edge-model-release
2020-02-18 16:06:18 emtf-featvis
2020-02-21 13:45:51 emtf-featvis-writable
2020-03-17 22:42:57 exphub-sdk
2019-08-27 11:36:18 exphub-tutorial-data
2019-08-27 14:18:17 face-attr-surveillance-released-modelrepo
2020-03-19 14:49:03 face-attribute-private-data-hhb
2019-10-09 11:29:11 face-recognition-noris-oss-temp
2021-04-07 18:44:04 faceattribute
2020-02-12 16:59:51 facerec-data-share
2019-09-25 10:25:43 facerec-demo
2020-03-03 14:27:31 facerec-raw-data-oss
2020-03-19 16:52:33 facerec-security-model
2021-03-10 22:31:20 fastai-data
2019-09-03 20:45:21 fastlabel-store
2020-02-21 13:53:04 featvis-report
2019-09-19 18:50:15 foxconn-conf
2020-02-12 17:00:00 gaohuazuo-share
2021-02-24 12:17:55 generalDetection
2020-02-12 17:00:00 ghlcelebrity
2020-10-28 13:09:53 giant-midi-dataset
2020-03-02 15:42:21 glassbase-share
2021-04-13 20:30:20 group-2-arch-dataset
2019-09-09 19:02:27 grpc-src
2020-02-12 17:45:37 heqi-data0-suffix
2019-08-27 11:41:45 heqi-share0-suffix
2021-05-11 15:26:35 hhb-shengjikang
2020-02-12 17:00:05 hilabel-data
2019-12-02 15:48:23 hubble-noris-oss
2020-01-16 11:48:23 hubble-temp-noris-oss
2019-12-25 17:23:34 hych-share
2020-05-24 23:07:40 inference-perfermance
2020-03-16 23:52:18 jhchen
2020-09-01 17:11:00 jinyanming
2021-04-22 10:54:13 jojoshare
2019-11-11 11:26:53 koala-share
2020-02-12 17:45:58 lhc-share
2021-03-09 20:49:40 libiao-bucket
2021-03-18 17:56:05 linzuzeng
2020-02-12 17:00:19 liudayuan-oss
2021-02-04 16:41:56 liuhongyun
2021-04-15 12:02:52 liuhongyun-hh-b
2020-10-22 19:08:51 liuli
2020-02-12 17:00:20 liuyibo-share
2019-10-22 15:09:06 liuyuxuan-public
2020-07-10 10:51:12 lixiaotong
2020-02-07 20:57:20 liyang-data-share
2019-12-10 15:42:58 liyang-share-oss
2020-11-26 01:37:11 lizemin-data-processing-oss
2021-03-15 14:32:58 lizemin-mm-models
2020-09-21 20:14:12 ljk-share
2021-02-22 18:34:05 llh
2019-10-08 15:41:46 lmkData
2020-02-12 17:46:22 lsy-share
2020-02-06 17:21:43 lvhaoran
2020-07-17 12:27:56 lyqtest
2020-02-12 17:46:29 megbrain-dist
2020-05-25 17:46:12 megbrain-integraion-test
2019-09-19 22:45:33 megdl-doc
2020-02-18 07:57:50 megengine-built
2019-12-05 17:26:27 megengine-doc
2020-03-11 14:18:34 metric-learning-datasets
2020-03-11 14:16:44 metric-learning-models
2020-02-12 17:00:29 metric-learning-runs
2019-12-17 13:58:18 model-zoo
2020-02-12 17:46:36 muhaoyuan-oss-share
2020-09-29 20:47:42 myc-share
2020-02-12 17:00:32 next-offline-tester
2019-12-28 11:04:01 nndenoise-badcase
2020-02-12 17:00:33 nnie-feature
2020-02-12 17:46:42 objects365-dataset
2019-08-20 15:53:43 oss-cz
2019-10-28 13:34:41 oss-fanshuyi
2019-08-20 17:34:39 osswfy
2020-12-25 13:54:38 ossxkx
2020-03-27 11:30:46 pages-oss
2020-03-03 16:39:40 paper-reading
2019-08-21 13:33:43 public-dataset-bj
2019-08-21 13:59:22 public-datasets-bj
2020-03-17 16:22:38 public-datasets-contrib
2019-08-26 22:48:31 public-datasets-contrib-dev
2019-08-19 18:40:18 public-datasets-weixin
2021-01-28 16:45:09 public-share
2020-08-28 16:58:11 pymegsdk-data
2021-03-31 16:36:36 qinyisong0a
2020-02-12 17:46:56 qlm-share
2020-11-15 14:04:46 r2e
2020-03-05 17:07:09 readthedocs
2020-03-10 11:01:19 reform-conf
2020-08-03 10:45:36 reprocess-conf
2006-01-02 23:04:05 research-model
2019-12-11 10:53:18 retinanet
2019-08-27 17:18:32 rjob-docs
2020-12-16 14:11:18 robotarm-docs
2021-05-18 07:20:29 rootfs
2006-01-02 23:04:05 share-oss
2020-02-12 17:52:27 share-yy
2021-02-07 15:50:45 shengjikang-data-processing-oss
2021-03-24 16:04:58 snapdet
2020-12-03 16:37:19 snapdet-benchmark-tmp
2020-11-26 22:52:19 snapdet-cache
2020-10-25 15:32:31 snapdet-resource
2020-10-28 17:45:18 snapdet-tmp
2021-03-18 16:14:43 speech-db
2020-03-19 14:40:46 srj-data-wh
2020-02-12 17:00:52 staging-aaa
2020-02-12 17:00:52 staginga-31a
2019-10-26 13:32:20 sunbin-share
2020-03-19 11:38:03 superIQ-public
2020-04-15 14:55:43 supershow2binary
2020-07-04 22:29:19 supervq-isp-simulate
2019-08-21 18:52:30 surveillance-attribute-dataset
2019-08-21 17:46:39 surveillance-attribute-dataset-rc
2021-02-02 14:41:29 sxq-tools
2020-07-28 11:04:33 t-chaiyizhe001-data-processing-oss
2020-08-13 19:47:15 t-jxh-data-pro-oss
2020-07-27 11:25:08 t-shengjikang-data-processing-oss
2020-05-06 17:59:10 temp-data-ckp
2021-05-14 19:32:43 test
2020-07-14 21:24:39 tets
2020-02-12 17:01:02 tianzhongbo-shared
2020-06-06 23:16:19 torchvision-data-shared
2020-02-12 17:52:49 ttt
2020-07-15 17:28:37 txh
2020-02-12 17:52:51 vehicledata
2021-02-03 14:23:10 video-ai-data
2019-12-13 18:22:04 videos-oss
2020-10-21 10:34:53 w5k-snapshot
2019-10-31 12:37:31 wangbin
2020-02-12 17:52:55 wangfengdata
2020-09-29 21:09:33 wangheng
2020-02-12 17:52:56 wangjianfengShare
2021-01-26 16:53:48 wangju
2020-03-19 15:17:48 wanglulu-oss
2020-03-19 15:18:12 wanglulu-oss-wha
2021-04-06 18:05:45 wangningzi
2020-03-03 17:33:35 wangsu-all-visited
2020-01-07 18:46:24 weixin-cluster
2020-11-23 17:44:46 weixin-det
2019-10-18 02:33:48 weixin-public-share
2020-10-09 18:25:57 wenhongbo-public
2020-05-07 12:34:13 wh-llcv-dataspace
2020-05-07 12:36:22 wh-llcv-train-log
2020-02-12 17:53:02 wheel
2020-03-09 15:17:46 wjn-share
2020-03-10 11:41:10 wq-share
2020-08-21 16:30:26 ws0a
2020-03-09 14:38:31 wsp-oss
2020-07-21 10:52:10 wuzhao
2020-07-16 16:24:31 wxy-data-sync
2020-02-12 17:53:09 wyz-share
2020-03-19 15:27:39 wzc-share
2020-06-26 21:32:05 xby
2021-02-22 13:33:07 xiechun-data-processing-oss
2019-09-10 11:11:34 xieyanbo-public
2020-02-12 17:01:15 xiongfu-taotiedata
2021-02-03 12:03:37 xkx-share
2019-10-22 10:39:32 xq-share
2020-05-27 17:21:50 xufei-testaaa
2020-08-14 15:04:38 y1xin
2021-02-26 15:36:25 yangbinchao
2020-09-11 23:55:16 yangl-share
2021-03-01 14:52:18 yangpengfei02-data-processing-oss
2020-05-26 09:21:31 yangsu
2020-02-12 17:53:19 yanyongluan
2020-03-09 12:34:08 yht-share
2020-09-24 14:04:01 yixin-share
2020-10-10 11:52:23 yl-dataset
2021-01-18 17:49:10 yl-project
2020-11-18 16:59:40 yl-scp
2020-08-03 15:17:26 yuantianyuan
2019-10-21 14:30:17 yyl-share
2020-03-19 14:22:31 yzy-share
2019-10-30 18:28:16 zData-wh
2021-01-18 15:53:20 zengsihang
2020-02-12 17:53:27 zhangchaoht
2020-08-03 17:20:32 zhangchunyi
2020-07-10 14:38:43 zhangfan
2021-04-22 20:06:51 zhangfeng
2020-02-12 17:01:23 zhangjianglan-data-processing-oss
2021-03-25 21:20:13 zhanglaicai
2020-02-12 17:53:30 zhangshoukuiData
2020-03-19 14:12:23 zhangshoukuiShare
2020-05-06 16:32:32 zhangxinyu
2020-10-16 15:28:21 zhangxinyu-cluster
2020-11-24 15:53:57 zhangxinyu-temp
2020-02-12 17:53:33 zhangyishan-share
2020-12-23 11:51:56 zhangzheng-cluster-test
2020-11-13 17:17:13 zhangzheng-share
2020-03-03 00:51:00 zhoushihao-share
2020-11-04 11:24:45 ziyi
2020-02-12 17:53:41 zookeeper-results-temporary
2021-03-02 01:08:53 zouxingxing-data-processing-oss
2019-08-29 15:18:58 zskShare
2019-09-05 00:33:12 zxy-public-share
2019-11-21 23:20:22 zxy-tmp
2020-02-12 17:53:45 zyq
2019-09-12 11:10:15 zyq-data
2020-08-03 14:18:29 zyq-fs
2019-09-12 13:52:26 zyq-model
2019-11-04 18:06:10 zyq-ws
2020-02-10 10:38:19 zyq-ws-wha
2020-03-09 17:44:46 zyz
2021-01-05 17:08:54 zzf-bucket
现在的报错是下面这样的，是不是没有权限？
wangcong@pensive-agnesi:/data/wangcong_test1$ python3 test_nori.py
/usr/lib/python3/dist-packages/requests/init.py:80: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn’t match a supported version!
RequestsDependencyWarning)
19 10:33:01 MegSkull 4.10.0 initialized. Using MegBrain: 8.10.1
19 10:33:01 MegHair 4.7.0 initialized
Traceback (most recent call last):
File “/usr/local/lib/python3.6/dist-packages/refile/s3.py”, line 301, in s3_isfile
client.head_object(Bucket=bucket, Key=key)
File “/home/wangcong/.local/lib/python3.6/site-packages/botocore/client.py”, line 386, in _api_call
return self._make_api_call(operation_name, kwargs)
File “/home/wangcong/.local/lib/python3.6/site-packages/botocore/client.py”, line 705, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “test_nori.py”, line 9, in
datas = io.load(“s3://ai-cultivate/task1/pics.pkl”)
File “/usr/local/lib/python3.6/dist-packages/meghair/utils/io.py”, line 395, in load
with fopen(fname, ‘rb’) as fin:
File “/usr/local/lib/python3.6/dist-packages/refile/smart.py”, line 387, in smart_open
return SmartPath(path).open(mode, **options)
File “/usr/local/lib/python3.6/dist-packages/refile/smart_path.py”, line 85, in open
return self.path.open(mode, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/refile/s3_path.py”, line 98, in open
if not self.is_file():
File “/usr/local/lib/python3.6/dist-packages/refile/s3_path.py”, line 36, in is_file
return s3_isfile(self.path_with_protocol)
File “/usr/local/lib/python3.6/dist-packages/refile/s3.py”, line 305, in s3_isfile
raise error
refile.errors.S3UnknownError: Unknown error encountered: ‘s3://ai-cultivate/task1/pics.pkl’, error: botocore.exceptions.ClientError(‘An error occurred (403) when calling the HeadObject operation: Forbidden’), endpoint: ‘http://oss.i.brainpp.cn’
没有配置oss，并且没有相关的oss权限。配置并找相关人员加权限后已解决。感谢！
