MindSpore 2.2.14 Release Notes

Major Features and Improvements

Parallel

• [STABLE] Changed the communication group of the send/receive operator to the world group in the parallel pipeline to avoid creating redundant communication groups and reduce the memory required for communication.
• [STABLE] Optimize the compilation cache to reduce the graph conversion process of the loading cache and improve the compilation cache performance.
• [BETA] Pipeline parallel supports Interleave. Optimize the performance when mciro batch is small.
• [BETA] Optimize checkpoint transformation speed when using pipeline parallel, support single stage transform.

Profiler

• [BETA] Dynamically start and stop profiling. Users can collect profiling data in real time according to the training situation, reducing the amount of data collected.
• [BETA] Profiling the communication operator time-consuming matrix. Users can find cluster communication performance bottlenecks by analyzing the communication operator time-consuming matrix.

Dump

• [BETA] The statistical information saved by Dump records MD5 values, and users can determine small differences in tensor values through MD5 values.
• [BETA] Dump supports the float16 data type and supports users to locate float16 type operator accuracy issues.

Bug Fixes

• [I962EV] Fixed issue on CPU and GPU environment with cond input dimension of 4d, 5d, 6d, 7d and 8d.
• [I96E5R] Fixed the issue in the PyNative that the input of the Mul operator is NCHW format on the Ascend platform.
• [I96I5D] Fixed the issue of incorrect input type when calculating Scalar type in dynamic shape scenario.
• [I99QAB] Fixed the issue where asnumpy cannot correctly identify the bfloat16 tensor in some scenarios.
• [I9ADZS] Fixed the data timeout issue in network training due to inefficient dataset recovery in the fault recovery scenario.
• [I8Y9JT] Fixed the issue that some network training does not converge due to the incorrect execution sequence of the optimizer in some specific scenarios where the nn.SGD optimizer has a large loss_scale and a small weight_decay.

Contributors

Thanks goes to these wonderful people:

fary86, wanghenchang, haozhang, mengyuanli, emmmmtang, luoyang, zhupuxu, zhangyongxian, liuluobin, LLLRT, TuDouNi, hujiahui8, wangtongyu6, ligan, zhuguodong, yanghaoran, YingtongHu, liyejun, zjun, 徐永飞, chuht, 张树仁, 徐安越, DeshiChen, shenyaxin, liujunzhu, shunyuanhan, yuchaojie, yao_yf, 没有窗户的小巷, yeyunpeng2020, weiyang, KevinYi, hedongdong, zhouyaqiang0, Margaret_wangrui, zhanghaibo, moran, huangziling, 朱家兴, GuoZhibin, 李良灿, jiaxueyu, gaoyong10, Greatpan, 宦晓玲, melody, 俞涵, jiangshanfeng, XinDu, ling, caifubi, zhangyinxia, gengdongjie, Erpim, XianglongZeng, zhangminli, fengyixing, 冯一航, 黄勇, panzhihui, 胡彬, linqingke, wangshaocong

Contributions of any kind are welcome!

MindSpore Lite 2.2.14 Release Notes

Bug Fixes

• [I96PJC] An error is reported when the CLIP model in MS format is loaded through the MindSpore Lite Python API.

Contributors

Thanks goes to these wonderful people:

wangtongyu6, zhuguodong, 徐永飞, 徐安越, yeyunpeng2020, moran, XinDu, gengdongjie.

Contributions of any kind are welcome!

MindSpore 2.2.13 Release Notes

API Change

Add timeout environment variables in dynamic networking scenarios:

• MS_TOPO_TIMEOUT: Cluster networking phase timeout time in seconds.
• MS_CLUSTER_RETRY_NUM: Number of node's retrying registration during cluster networking phase.
• MS_NODE_TIMEOUT: Node heartbeat timeout in seconds.
• MS_RECEIVE_MSG_TIMEOUT: Node timeout for receiving messages in seconds.

Bug Fixes

• [I9CR96] Fix the issue of insufficient timeout time causing failure for dynamic networking startup in large-scale clusters.

Contributors

Thanks goes to these wonderful people:

ZPaC, limingqi107, lizhenyu, jiangshanfeng

Contributions of any kind are welcome!

MindSpore 2.2.12 Release Notes

Major Features and Improvements

• [STABLE] Optimize scenarios where network parameters are initialized by fp32, and optimizer parallel mode is on, reducing the amount of Cast operator.
• [STABLE] Add detection and processing capabilities to silent data corruption. Silent data corruptions may lead to error during training procedures, this helps users to prevent or lower the cost of fault location, which caused by silent data corruptions.

Bug Fixes

• [I97D1L] Fix ReduceLROnPlateau, LRScheduler, CosineAnnealingWarmRestarts dynamic learning rate related interface sample error.
• [I970HV] Fix the problem where order of AllGather/ReduceScatter between two cards is not preserved.
• [I99JPI] Fix load checkpoint for bfloat16 parameter during vague load mode.

Contributors

Thanks goes to these wonderful people:

yao_yf, YijieChen, 冯一航, yuchaojie, 李良灿, YuJianfeng, huangxinjing, GuoZhibin, looop5

Contributions of any kind are welcome!

MindSpore 2.3.0-rc1 Release Notes

Major Features and Improvements

DataSet

• [STABLE] Support integrity check, encryption and decryption check for MindRecord to protect the integrity and security of user data.
• [STABLE] MindRecord api changes: FileWriter.open_and_set_header is deprecated since it has been integrated into FilterWriter, if the old version code reports an error, delete this call; Add type checking for data in FileWriter to ensure that the data type defined by the Schema matches the real data type; The return value of all methods under Mindrecord are removed, replaced by an exception when processing error is occurred.
• [STABLE] Support Ascend processing backend for the following transforms: ResizedCrop, HorizontalFlip, VerticalFlip, Perspective, Crop, Pad, GaussianBlur, Affine.
• [STABLE] Optimized the content of data processing part in model migration guide, providing more examples to compare with third-party frameworks.
• [STABLE] Optimized the parsing efficiency of TFRecordDataset in multiple data columns scenario, improving the parsing performance by 20%.

PIJIT

• [BETA]PIJit analyzes and adjusts the Python bytecode and performs graph capture and graph optimization on the execution flow. Supported Python codes are executed in static graph mode, and unsupported ones are divided into subgraphs and executed in dynamic graph mode, automatically achieving dynamic and static unification. Users can enable the PIJit function by decorating the function with @jit(mode="PIJit", jit_config={options:value}).

Inference

• [DEMO] The integrated architecture of large model inference, upgrade, training, and promotion unifies scripts, distributed policies, and runtime. The period from training to inference deployment of typical large models is reduced to days. Large operators are integrated to reduce the inference latency and effectively improve the network throughput.

AutoParallel

• [STABLE] Add msrun startup method to launch distributed job with single instruction.
• [STABLE] Add to be deprecated hint for RankTable startup method.
• [STABLE] Eliminate redundant constants in graph mode to improve compilation performance and memory overhead.
• [STABLE] The subgraph scenario optimizer parallelizes the first subgraph inline, allowing some computation and communication masking under pipeline parallelism to be performed.
• [STABLE] Communication information export: export model communication information (communication domain, communication volume) during compilation, and input it to the cluster as the basis for communication scheduling.
• [STABLE] Pipeline parallel inference is optimized, eliminates shared weights forwarding between stages, improving execution performance. Supports automatic broadcast of pipeline inference results, improving the usability of autoregressive inference.
• [STABLE] Operator-level parallel sharding supports the configuration of the mapping between the device layout and tensor layout during MatMul/Add/LayerNorm/GeLU/BiasAdd operator sharding.
• [STABLE] Supports gradient communication and backward calculation overlapping in the data parallel dimension.
• [STABLE] Single device simulation compilation, used to simulate the compilation process of a certain device in multi device distributed training, assisting in analyzing the compilation processes and memory usage on the front and back ends.
• [STABLE] Implement ops.Tril sharding to reduce the memory and performance requirements on a single device.
• [BETA] Supports the fusion between communication operators and computing operators, in order to overlap communication overheads with computation and improve network performance.
• [BETA] Load checkpoints and compile graphs in parallel to accelerate fault recovery.

Runtime

• [BETA] Support O0/O1/O2 multi-level compilation to improve static graph debugging and tuning capabilities.

FrontEnd

• [STABLE] The framework supports the bfloat16 data type. dtype=mindspore.bfloat16 can be specified when a tensor is created.
• [STABLE] The syntax support capability of the rewrite component is optimized, syntaxs such as class variables, functions, and control flows can be parsed.
• [STABLE] New context setting: debug_level. User can use mindspore.set_context(debug_level=mindspore.DEBUG) to get more debug information.

Profiler

• [BETA] Dynamically start and stop profiling. Users can collect profiling data in real time according to the training situation, reducing the amount of data collected.
• [BETA] Profiling the communication operator time-consuming matrix. Users can find cluster communication performance bottlenecks by analyzing the communication operator time-consuming matrix.
• [BETA] Improve the performance of Ascend environment in parsing profiling data.
• [BETA] Supports offline analysis of data generated by Profiling. Users can collect data first and then parse the data as needed.
• [BETA] Supports collecting performance data of HBM, PCIe, and l2_cache to enrich performance analysis indicators.

Dump

• [BETA] The statistical information saved by Dump records MD5 values, and users can determine small differences in tensor values through MD5 values.
• [BETA] Dump supports the float16 data type and supports users to locate float16 type operator accuracy issues.

PyNative

• [STABLE] Reconstruct the single operator calling process for dynamic graphs to improve the performance of dynamic graphs.

Ascend

• [BETA] Support set configuration options of CANN, which are divided into two categories: global and session. Users can configure them through mindspore.set_context(Ascend_configuration={"ge_options": {"global": {"global_option": "option_value"}, "session": {"session option": "option_value"}}).

API Change

• Add mindspore.hal API to support stream, event, and device management capabilities.
• Add mindspore.multiprocessing API to provide the capability of creating multiple processes.

Operators

• [BETA] mindspore.ops.TopK now supports the second input k as an int32 type tensor.

Bug Fixes

• [I92H93] Fixed the issue of 'Launch kernel failed' when using the Print operator to print string objects on the Ascend platform.
• [I8S6LY] Fixed RuntimeError: Attribute dyn_input_sizes of Default/AddN-op1 is [const vector]{}, of which size is less than 0 error of variable-length input operator, such as AddN or Concat, for dynamic shape process in graph mode on the Ascend platform.
• [I9ADZS] Fixed the data timeout issue in network training due to inefficient dataset recovery in the fault recovery scenario.

Contributors

Thanks goes to these wonderful people:

AlanCheng511，AlanCheng712，bantao，Bingliang，BJ-WANG，Bokai Li，Brian-K，caifubi，cao1zhg，CaoWenbin，ccsszz，chaiyouheng，changzherui，chenfei_mindspore，chengbin，chengfeng27，chengxb7532，chenjianping，chenkang，chenweifeng，Chong，chuht，chujinjin，Cynthia叶，dairenjie，DavidFFFan，DeshiChen，douzhixing，emmmmtang，Erpim，fangzhou0329，fary86，fengxun，fengyixing，fuhouyu，gaoshuanglong，gaoyong10，GaoZhenlong，gengdongjie，gent1e，Greatpan，GTT，guoqi，guoxiaokang1，GuoZhibin，guozhijian，hangq，hanhuifeng，haozhang，hedongdong，hejianheng，Henry Shi，heyingjiao，HighCloud，Hongxing，huandong1，huangbingjian，HuangLe02，huangxinjing，huangziling，hujiahui8，huoxinyou，jiangchenglin3，jianghui58，jiangshanfeng，jiaorui，jiaxueyu，JichenZhao，jijiarong，jjfeing，JoeyLin，JuiceZ，jxl，kairui_kou，kate，KevinYi，kisnwang，lanzhineng，liangchenghui，LiangZhibo，lianliguang，lichen，ligan，lihao，limingqi107，ling，linqingke，liruyu，liubuyu，liuchao，liuchengji，liujunzhu，liuluobin，liutongtong9，liuzhuoran2333，liyan2022，liyejun，LLLRT，looop5，luochao60，luojianing，luoyang，LV，machenggui，maning202007，Margaret_wangrui，MaZhiming，mengyuanli，MooYeh，moran，Mrtutu，NaCN，nomindcarry，panshaowu，panzhihui，PingqiLi，qinzheng，qiuzhongya，Rice，shaojunsong，Shawny，shenwei41，shenyaxin，shunyuanhan，silver，Songyuanwei，tangdezhi_123，tanghuikang，tan-wei-cheng，TingWang，TronZhang，TuDouNi，VectorSL，WANG Cong，wang_ziqi，wanghenchang，wangpingan，wangshaocong，wangtongyu6，weiyang，WinXPQAQ，wtcheng，wudawei，wujiangming，wujueying，wuweikang，wwwbby，XianglongZeng，xiaosh，xiaotianci，xiaoxin_zhang，xiaoxiongzhu，xiaoyao，XinDu，xingzhongfan，yanghaoran，yangluhang，yangruoqi713，yangzhenzhang，yangzishuo，yanjiaming，Yanzhi_YI，yao_yf，yefeng，yeyunpeng2020，yide12，YijieChen，YingLai Lin，YingtongHu，youshu，yuchaojie，YuJianfeng，zangqx，zby，zhaiyukun，zhangdanyang，zhanghaibo，zhanghanLeo，zhangminli，zhangqinghua，zhangyanhui，zhangyifan，zhangyinxia，zhangyongxian，ZhangZGC，zhanzhan，zhaoting，zhengyafei，zhengzuohe，ZhihaoLi，zhouyaqiang0，zhuguodong，zhumingming，zhupuxu，zichun_ye，zjun，zlq2020，ZPaC，zuochuanyong，zyli2020，陈宇，代宇鑫，狄新凯，范吉斌，冯一航，胡彬，宦晓玲，黄勇，康伟，李良灿，李林杰，刘崇鸣，刘力力，刘勇琪，吕浩宇，没有窗户的小巷，王禹程，吴蕴溥，熊攀，徐安越，徐永飞，许哲纶，俞涵，张峻源，张树仁，张王泽，张栩浩，郑裔，周莉莉，周先琪，朱家兴，邹文祥

Contributions of any kind are welcome!

MindSpore 2.2.11 Release Notes

Major Features and Improvements

scipy

• [Stable] Add new API mindspore.scipy.optimize.linear_sum_assignment in scipy module to solve the linear sum assignment problem. It can find the least-cost assignment based on a given cost matrix.

Bug Fixes

• [I8JVRU] Fixed the problem where the results of the bernoulli random operator running twice on the GPU are probabilistically consistent.
• [I8OC32] Fixed the segmentation fault error because the MatrixSetDiagV3 operator does not verify abnormal input.

Contributors

Thanks goes to these wonderful people:

fary86, wanghenchang, haozhang, mengyuanli, emmmmtang, luoyang, zhupuxu, zhangyongxian, liuluobin, LLLRT, TuDouNi, hujiahui8, wangtongyu6, ligan, zhuguodong, yanghaoran, YingtongHu, liyejun, zjun, 徐永飞, chuht, 张树仁, 徐安越, DeshiChen, shenyaxin, liujunzhu, shunyuanhan, yuchaojie, yao_yf, 没有窗户的小巷, yeyunpeng2020, weiyang, KevinYi, hedongdong, zhouyaqiang0, Margaret_wangrui, zhanghaibo, moran, huangziling, 朱家兴, GuoZhibin, 李良灿, jiaxueyu, gaoyong10, Greatpan, 宦晓玲, melody, 俞涵, jiangshanfeng, XinDu, ling, caifubi, zhangyinxia, gengdongjie, Erpim, XianglongZeng, zhangminli, fengyixing, 冯一航, 黄勇, panzhihui, 胡彬, linqingke, wangshaocong

Contributions of any kind are welcome!

MindSpore Lite 2.2.11 Release Notes

Bug Fixes

• [I8TPLY] Fixed SSD MobileNetV2 FPN network inference error on Atlas inference series products(configured with Ascend 310P AI processor).

Contributors

Thanks goes to these wonderful people:

wangtongyu6, zhuguodong, 徐永飞, 徐安越, yeyunpeng2020, moran, XinDu, gengdongjie.

Contributions of any kind are welcome!

MindSpore 2.2.1 Release Notes

Bug Fixes

• [I7R3R5] Fixed the problem that the network precision of the ResNet-50 on the Ascend platform deteriorates.
• [I8A9RH] Fixed an issue where the DBNet(ResNet-50) network precision on the Ascend platform deteriorates.
• [I8B8IW] Fixed the segment error caused by out-of-bounds multi-dimensional tensor assignment.
• [I8J0F4] Fixed an issue where the multidimensional Tensor extension dimension fails to be executed in the dynamic graph.
• [I87P3P] Fixed an issue where the compilation cache fails to be loaded during secondary training on the Ascend platform.
• [I86GP9] Fixed an issue where the UNet3D network inference precision deteriorates on the Ascend platform.
• [I89B4K] Fixed an issue where the dynamic rank execution of dynamic graphs on the Windows platform is suspended.
• [I8CX0C] Fixed an issue where dynamic images occasionally fail in mixed precision mode on the Ascend platform.
• [I8BGCF] Fixed an issue where a segment error occurs when the command is executed in dynamic diagram mode of the AirNet network on the Ascend platform.
• [I8L5DS] Fixed an issue where the ResNet-50 image segmentation network dynamic image is executed slowly on the Ascend platform.

Contributors

Thanks goes to these wonderful people:

yufan, dingcheng, lvzhangcheng, zhunaipan, fangwenyi, weiyang, changzherui, chujinjin, zangqingxiang, yuchaojie, wuweikang, tanghuikang, xiaoyao, huangbinjian, zhoupeichen, chenfei_mindspore, hedongdong, wangnan, zhengzuohe, yanghaoran, zouliqin, luoyang, liuchongmin, lujiale, machenggui, wangcong, lixiangyi, wangting, huangyong

Contributions of any kind are welcome!

MindSpore Lite 2.2.1 Release Notes

Bug Fixes

• [I88055] Fixed a function issue caused by incorrect format setting of the gridsample operator in MindSpore Lite inference.
• [I8D80Y] The MindSpore Lite inference single-operator invoking process resources are not released and exits abnormally.

Contributors

Thanks goes to these wonderful people:

zhanghaibo, wangsiyuan, wangshaocong, chenjianping

Contributions of any kind are welcome!

MindSpore 2.2.0 Release Notes

Major Features and Improvements

DataSet

• [STABLE] The row_size parameter of data operation map/batch is extended to support passing list, which stands for [Input Shared Memory, Output Shared Memory], so as to flexibly control the size of shared memory in multi-process mode.
• [STABLE] Provide 100% mindspore.dataset and mindspore.dataset.transforms samples for reference.
• [STABLE] ConcatDataset supports global sampling. After combining data from multiple sources using concat operation, data can be globally sampled randomly to enhance data diversity.
• [STABLE] When the model.train API is used for training, TimeMonitor(.., data_time=True) can be used to monitor data processing performance in real time.
• [STABLE] Introduced the jemalloc library to solve the problem of slow memory rise due to untimely memory debris recovery in extreme scenarios.

FrontEnd

• [STABLE] Support adding decorator @lazy_inline to make a graph generated from cell being inlined lazily, which can improve the compilation performance effectively.
• [STABLE] Optimize the function of mixed precision training, support automatic rewriting of Python scripts through rewrite to achieve mixed precision strategies, and support automatic parsing of functions, branch statements, and other syntax.
• [STABLE] Mixed precision function optimization, ReWrite supports syntax parsing of class functions and branch statements, and extends O1 functionality.
• [STABLE] Optimize the dynamic learning rate function and add APIs such as MultiStepLR; function get_lr and global_step decoupling, extending optimizer module functionality.
• [STABLE] Optimize API code samples, API difference tables, and tutorials for using higher-order functions.

Operator

• [STABLE] Add new operator primitive mindspore.ops.Dense.
• [STABLE] Add the random number operator state management feature, which allows the random number operator to save the state of the random number, and can be stably reproduced in scenarios such as model parallelism and recalculation. Currently, it only supports CPU/GPU platforms, and the involved random number operators include: mindspore.ops.Multinomial, mindspore.ops.MultinomialWithReplacement, mindspore.ops.ParameterizedTruncatedNormal, mindspore.ops.StandardLaplace, mindspore.ops.StandardLaplace, mindspore.ops.Uniform, mindspore.ops.UniformInt, mindspore.ops.UniformReal, mindspore.ops.UniformInt, mindspore.ops.Dropout, mindspore.ops.RandomChoiceWithMask, mindspore.ops.RandomCategorical, mindspore.ops.RandomShuffle, mindspore.ops.RandamGamma, mindspore.ops.RandomPoisson and mindspore.ops.TruncatedNormal.
• [STABLE] When a GPU operator encounters an illegal input scenario, it supports asynchronously printing error logs in the CUDA kernel of the operator to the Host side and interrupting the execution of the current CUDA Stream, improving the efficiency of user operator problem positioning.

PyNative

• [STABLE] Support viewing mechanism in PyNative mode.
• [STABLE] Function enhancement in PyNative mode: sens supports dict input type.

Ascend

• [STABLE] Supports user configurable operator high-precision/high-performance mode, users can use context.set_context(ascend_config={"op_precision_mode": "/path/to/op_precision_config_file"}) to configure high-precision/high-performance modes for some TBE operators.
• [BETA] Supports user configurable operators for fp16-in and fp32-out, users can use context.set_context(ascend_config={"precision_mode": "force_fp32"}) to configure fp16-in and fp32-out for the TBE Cube operators.
• [BETA] Remove the strong binding between jit_level="O3" and GE processes, so users no longer need to set jit_level="O3" when executing GE processes.

Parallel

• [STABLE] Support the gradient accumulation feature in non-pipeline parallel scenarios in semi-automatic/fully automatic mode. Users can enable gradient accumulation by writing net = GradAccumulationCell(net, micro_size). The gradient accumulation feature is compatible with the lazy_inline feature.

Inference

Since version 2.2, the MindSpore main release package does not provide the inference interface enabling for the Ascend 310. If you need to use the inference interface, install the MindSpore Lite release package or download the MindSpore version earlier than 2.0. For details about how to install and use MindSpore Lite, see https://www.mindspore.cn/lite/en. HUAWEI Ascend 310 (Ascend) is an energy-efficient and highly integrated AI processor for edge scenarios. It supports inference on MindIR models. In the earlier version, MindSpore provides two methods for enabling inference on the Ascend 310 hardware:

1. The MindSpore main release package provides the matching Ascend 310 version that supports C++ inference interfaces.
2. The MindSpore Lite release package provides the matching Ascend version and supports C++ and Java inference.

The C++ APIs provided by the two solutions are basically the same. In the future, MindSpore Lite is used instead of building and maintaining two sets of interfaces. The original 310 inference service built based on the MindSpore main release package can be switched to MindSpore Lite with a few modifications. For details, see https://www.mindspore.cn/docs/en/r2.2/faq/inference.html.

Bug fixes

• [I7SDA0] Fixed an issue where the accuracy of the CRNN network deteriorates on the NES platform.
• [I7T4QK] Fixed an issue where the inference precision of the WGAN network deteriorates on the OptiX OSN 8800 platform.
• [I7TJ8Z] Fixed an issue where the inference precision of the LGTM network deteriorates on the OptiX OSN 8800 platform.
• [I7M58O] Fixed ASR-dynamic network training core dump issue on Ascend platform.
• [I7L6B6] Fixed an issue where child processes do not exit in some scenarios when dataset is in multi-process mode.
• [I7L7AE] Fixed an issue where dataset pipeline contains repeat operations and dynamic batchinfo.get_epoch_num() is incorrectly used in dataset.batch.
• [I7UY7G] Rectify the file permission modification error in OBSMindDataset.

Contributors

Thanks goes to these wonderful people:
bantao, Bingliang, BJ-WANG, Brian-K, caifubi, ccsszz, changzherui, chenfei_mindspore, chengfeng27, chenhaozhe, chenjianping, chenkang, chenweifeng, chuht, chujinjin, CShu0507, Cynthia叶, DeshiChen, douzhixing, Erpim, Etienne, fary86, fengxun, fengyixing, gaoshuanglong, Gaoxiong, gaoyong10, GaoZhenlong, Greatpan, GuoZhibin, guozhijian, hangq, hanhuifeng, haozhang, hedongdong, Henry Shi, HighCloud, Hongxing, huangbingjian, huanghui, huangxinjing, huangziling, hujiahui8, huoxinyou, HWalkingMan, jianghui58, jiangshanfeng, jiaorui, jijiarong, jjfeing, JuiceZ, jxl, KevinYi, kisnwang, KXiong, lanzhineng, Li Qingguo, LiangZhibo, lianliguang, ligan, lihao, Lihoon, limingqi107, ling, linqingke, liruyu, liubuyu, liuchao, liujunzhu, liuluobin, liupeng303, liutongtong9, liyan2022, liyejun, looop5, luochao60, luojianing, luoyang, machenggui, maning202007, Margaret_wangrui, MaZhiming, mengyuanli, moran, NaCN, nomindcarry, panshaowu, panzhihui, qinzheng, qiuzhongya, r1chardf1d0, shaojunsong, shenwei41, shenyaxin, shenzhangyi, Shira Zaloshinski, shunyuanhan, tangdezhi_123, tanghuikang, tan-wei-cheng, tan-wei-cheng-3260, TronZhang, TuDouNi, VectorSL, wang_ziqi, wanghenchang, wangpingan, wangshaocong, wangtongyu6, wtcheng, wujueying, XianglongZeng, xiaotianci, xiaoxin_zhang, xiaoxiongzhu, xiaoyao, xiaoyuanyuan, XinDu, xujinliang, xupan, yanghaoran, yangluhang, yangruoqi713, yangsijia, yangzhenzhang, yangzishuo, yanjiaming, Yanzhi_YI, yao_yf, yefeng, yeyunpeng2020, yide12, YijieChen, YingLai Lin, YingtongHu, yonibaehr, youshu, yuchaojie, YuJianfeng, zangqx, zhaizhiqiang, zhangbuxue, zhangchunlei, zhangdanyang, zhangdong, zhanghaibo, zhangminli, zhangqi, zhangqinghua, zhangyanhui, zhangyifan, zhangyongxian, zhangzhen, zhangzheng, zhanzhan, zhengzuohe, ZhihaoLi, zhoufeng, zhouyaqiang0, zhuguodong, zhupuxu, zichun_ye, zjun, ZPaC, zuochuanyong, zyli2020, 陈宇, 程超, 范吉斌, 冯浩, 冯一航, 胡彬, 宦晓玲, 黄勇, 雷元哲, 黎冠新, 李良灿, 李林杰, 刘崇鸣, 刘力力, 刘思铭, 刘勇琪, 吕浩宇, 没有窗户的小巷, 沈竞兴, 王禹程, 王振邦, 徐安越, 徐永飞, 俞涵, 张澍坤, 周超, 朱家兴

Contributions of any kind are welcome!

MindSpore Lite 2.2.0 Release Notes

Major Features and Improvements

FlashAttention Operator Fusion

• [STABLE] The OptiX OSN Ascend 910 series supports the FlashAttention large operator fusion of the LLAMA and stable diffusion models.

MindSpore 2.1.1 Release Notes

Bug fixes

• [I7Q9RX] The Ascend platform supports adaptive identification of different hardware types.
• [I7SDA0] Fixed an issue where the accuracy of the CRNN network deteriorates on the NES platform.
• [I7T4QK] Fixed an issue where the inference precision of the WGAN network deteriorates on the OptiX OSN 8800 platform.
• [I7TJ8Z] Fixed an issue where the inference precision of the LGTM network deteriorates on the OptiX OSN 8800 platform.

Contributors

Thanks goes to these wonderful people:

changzherui, chenfei_mindspore, chenjianping, chenkang, chenweifeng, chujinjin, fangwenyi, GuoZhibin, guozhijian, hangq, hanhuifeng, haozhang, hedongdong, You Shu, Zhou Feng, Dai Yuxin

Contributions of any kind are welcome!

MindSpore 2.2.10 Release Notes

Major Features and Improvements

Operators

• [STABLE] FastGelu, BatchMatMul, AllReduce, AllGather, Broadcast, ReduceScatter support bfloat16 data type
• [STABLE] AllGather support uint8 data type

Bug Fixes

• [I8ALW3] Fixed networks including Faster R-CNN, DeepText, MaskRCNN-ResNet50, which had errors while training RandomChoiceWithMask operator in Ascend 910 8P scenario.
• [I8LKG7] Fixed graph compilation error of UNet-2D in Ascend 910 1P/8P scenario.
• [I8KU3X] Fixed CRNN-ResNet34 network, which stuck in training phase in Ascend 910 1P/8P PyNative mode.
• [I8KTHH] Fixed BERT network error when training without allreduce grouped fusion with enable_parallel_optimizer=True, in Ascend 910 8P scenario.

Contributors

Thanks goes to these wonderful people:

李林杰, TuDouNi, chengxb7532, Henry Shi, rms-infer-type, 朱家兴, zhouyaqiang0, tanghuikang, gaoyong10, gengdongjie, yao_yf, hujiahui8, hanhuifeng, shenyaxin, KevinYi, 冯一航, chengfeng27, JuiceZ, zhangyanhui, jijiarong, xiaoxiongzhu, 没有窗户的小巷, ling, liyan2022, haozhang, zangqx, xiaoyao, liujunzhu, 胡彬, panzhihui, wangshaocong, linqingke, jianghui58, qiuzhongya, yangruoqi713, zhangminli, moran, 王禹程, shaojunsong, wangtongyu6, zhupuxu, luoyang, 徐安越, qinzheng, caifubi, 徐永飞, chenkang, youshu, XinDu, liubuyu, jxl, yeyunpeng2020, huoxinyou, yefeng, jiaorui, wangpingan, cao1zhg, zjun, zyli2020, yanjiaming, Cynthia叶, 胡安东, 李良灿, liruyu, liuluobin, lihao, huangbingjian, YijieChen, jjfeing, looop5, 刘力力, xiaoxin_zhang, yangluhang, chenweifeng, jiangshanfeng, zichun_ye, 陈宇, NaCN, ligan, YingLai Lin, huangziling, chenjianping, DeshiChen, chengbin, kairui_kou, ccsszz, yanghaoran, zhangdanyang, Yanzhi_YI, zhengzuohe, hangq, TronZhang, wanghenchang, HighCloud, 吕浩宇, VectorSL, ZPaC, mengyuanli, maning202007, 刘勇琪, r1chardf1d0, fary86, 刘崇鸣, yuchaojie, douzhixing, fengyixing

Contributions of any kind are welcome!

MindSpore Lite 2.2.10 Release Notes

Bug Fixes

• [I8K7CC] Optimize error message when non-string segments are passed to get_model_info.

Contributors

Thanks goes to these wonderful people:

gengdongjie, zhangyanhui, xiaoxiongzhu, wangshaocong, jianghui58, moran, wangtongyu6, 徐安越, qinzheng, 徐永飞, youshu, XinDu, yeyunpeng2020, yefeng, wangpingan, zjun, 胡安东, 刘力力, 陈宇, chenjianping, kairui_kou, zhangdanyang, hangq, mengyuanli, 刘崇鸣

Contributions of any kind are welcome!

MindSpore 2.1.1 Release Notes

Bug fixes

• [I7Q9RX] 昇腾平台支持不同硬件类型自适应识别。
• [I7SDA0] 修复了昇腾平台上CRNN网络精度劣化的问题。
• [I6QYCD] 修复了昇腾平台上maskrcnn网络精度劣化问题。
• [I7T4QK] 修复了昇腾平台上wgan网络推理精度劣化问题。
• [I7TJ8Z] 修复了昇腾平台上lgtm网络推理精度劣化问题。

贡献者

感谢以下人员做出的贡献:

changzherui,chenfei_mindspore,chenjianping,chenkang,chenweifeng,chujinjin,fangwenyi,GuoZhibin,guozhijian,hangq,hanhuifeng,haozhang,hedongdong,尤澍,zhoufeng,代宇鑫

欢迎以任何形式对项目提供贡献！

MindSpore Lite 2.1.1 Release Notes

Major Features and Improvements

• [STABLE] MindSpore Lite Cloud Inference adds support for Python 3.8 and Python 3.9

MindSpore 2.1.0 Release Notes

Major Features and Improvements

FrontEnd

• [BETA] JIT Fallback supports variable scenarios. In static graph mode, JIT Fallback supports return of Dict type and Scalar type, supports property setting of non-Parameter type objects, supports partial in-place modification operations of List, and supports third-party libraries such as NumPy. Moreover, it supports related operations of user-defined classes and supports Python basic operators and built-in functions to use more data types. It is compatible with features like control flow, side effects, automatic differentiation. For more details, please refer to Static Graph Syntax Support.

• [BETA] In static graph mode, the error message of using undefined variables in the control flow scene is optimized. When using variables defined in if, while, and for control flow branches, the variables need to be initialized and defined before the control flow.

• [STABLE] Add module ReWrite, support the ability to modify multiple network in batches based on customized rules.

• [BETA] Add optim_ex module for optimizers, extend the current functionality, support parameter grouping for every parameter in the optimizer, and support parameter modification by assignment while training.

• [STABLE] Optimize PyTorch and MindSpore API Mapping Table, specify the differences between APIs among functionality, parameter, input, output and specialized cases.

PyNative

• Optimize the performance of dynamic shape scenes in PyNative mode.

DataSet

• [STABLE] Optimize the memory structure of MindRecord data files. Memory consumption can be reduced 60% when loading 100TB+ data for training.
• [STABLE] Support single-thread execution of data processing pipeline, and users can add code in the data pipeline for debugging.
• [STABLE] Optimize the performance of TFRecordDataset to improve the performance of dataset loading by 60%+. Optimize the performance of batch to improve the performance by 30% for the scenarios with large number of batch.
• [STABLE] Optimize API documentation of mindspore.dataset and mindspore.dataset.transforms. Four new sample libraries have been added to show the effect of data enhancement, namely: Load & Process Datasets Using Data Pipeline, Visual Transformation Sample Library, Text Transform Sample Library, Audio Transform Sample Library

AutoParallel

• [STABLE] Support offload parameters or intermediate activations to the CPU or NVMe storage during training process. Users can enable this offload feature by configuring context to scale up the trainable model size.

• [STABLE] Enhanced automatic parallel capability including:

  1. Performance of automatic strategy for typical networks is no less than 90% of default configuration.

  2. Support 3D hybrid parallel training: automatic operator-level strategy generation combined with manual configured pipeline partition.

Runtime

• [STABLE] Upgrade OpenMPI version to 4.1.4.
• [STABLE] Upgrade NCCL version to 2.16.5.
• [STABLE] Assign rank id continuously in same node when using dynamic cluster to launch distributed jobs.
• [STABLE] No adaptation code is required for Scheduler node. The script of Scheduler could be identical to that of Worker.

Ascend

• [STABLE] Support dump assisted debug information for operator AIC Error scenario. The information includes the operator task name, stream ID, input/output/workspace address and so on.
• [STABLE] Provide default processing mechanism, which skips its execution, for CANN operators for empty Tensor output scenarios.
• [STABLE] Supplement debug information when network model fails to execute in graph mode. The debug information will saved in a CSV file in rank_${id}/exec_order/, recording the task ID and stream ID of each task.

Profiler

• [STABLE] The Profiler supports the collection of time-consuming data from all phases on the Host side.
• [BETA] The Profiler supports the collection of memory data from all phases on the Host side.
• [BETA] The Profiler supports the collection of data processing operator time consumption.

API Change

• mindspore.dataset.GraphData, mindspore.dataset.Graph, mindspore.dataset.InMemoryGraphDataset, mindspore.dataset. ArgoverseDataset are no longer evolved and are deprecated. Use MindSpore Graph Learning for related functional replacements. When replacing networks in Model repositories that use this API, please refer to GCN for GCN and GAT.
• mindspore.set_context adds jit_syntax_level option, which is used to set JIT syntax support level. For more details, please refer to set_context.
• Change the model.infer_predict_layout interface, which has a new parameter skip_backend_compile with a default value of False. Set to True when the user wants to skip the backend compilation process to get the parameter slicing strategy.

Operators

• Add operator primitive for mindspore.ops.ApplyAdamWithAmsgradV2. It is recommended to call this operator through API mindspore.nn.Adam.
• Add operator primitive for mindspore.ops.UpsampleTrilinear3D. It is recommended to call this operator through API mindspore.ops.interpolate.
• Add operator primitive for mindspore.ops.UpsampleNearest3D. It is recommended to call this operator through API mindspore.ops.interpolate.

API Deprecation

• Deprecate operator primitive mindspore.ops.ScatterNonAliasingAdd. It is recommended to use operator primitive mindspore.ops.TensorScatterAdd as a replacement.

Backwards Incompatible Change

• Interface name: mindspore.nn.Dense, mindspore.nn.Conv1d, mindspore.nn.Conv1dTranspose, mindspore.nn.Conv2d, mindspore.nn.Conv2dTranspose, mindspore.nn.Conv3d, mindspore.nn.Conv3dTranspose

  Changes: Change initialization parameter strategy. The default value of weight_init is changed from "normal" to None, and the default value of bias_init is changed from "zeros" to None.

  Description: The default initialization method for weights has been changed from "normal" to internal HeUniform initialization. The default initialization method of bias is changed from "zeros" to internal Uniform initialization.

  Original interface	v2.1 interface
  mindspore.nn.Dense(in_channels, out_channels, weight_init='normal', bias_init='zeros', has_bias=True, activation=None)	mindspore.nn.Dense(in_channels, out_channels, weight_init=None, bias_init=None, has_bias=True, activation=None)
  mindspore.nn.Conv1d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')	mindspore.nn.Conv1d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init=None, bias_init=None)
  mindspore.nn.Conv1dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')	mindspore.nn.Conv1dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init=None, bias_init=None)
  mindspore.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCHW')	mindspore.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init=None, bias_init=None, data_format='NCHW')
  mindspore.nn.Conv2dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, output_padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros')	mindspore.nn.Conv2dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, output_padding=0, dilation=1, group=1, has_bias=False, weight_init=None, bias_init=None)
  mindspore.nn.Conv3d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCDHW')	mindspore.nn.Conv3d(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, has_bias=False, weight_init=None, bias_init=None, data_format='NCDHW')
  mindspore.nn.Conv3dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, output_padding=0, has_bias=False, weight_init='normal', bias_init='zeros', data_format='NCDHW')	mindspore.nn.Conv3dTranspose(in_channels, out_channels, kernel_size, stride=1, pad_mode='same', padding=0, dilation=1, group=1, output_padding=0, has_bias=False, weight_init=None, bias_init=None, data_format='NCDHW')

Bug Fixes

• [I6TKLW] Fix the issue of MobileNetV2 network performance degradation on the Ascend platform.
• [I7CP5H] Fix the issue where ASR network training failed on the Ascend platform.
• [I6QYCD] Fix the issue where the BERT-Large-Boost network fails to train in pynative mode on the Ascend platform.
• [I7I3EZ] Fix the issue that caused run_check() failure due to changes to the enumeration interface in Pillow version 10.0.0. If encountered in a lower version of MindSpore, install versions of Pillow below 10.0.0 to avoid this issue.
• [I7IZ8K] Fix accuracy issues with the assignsub interface in PyNative mode.
• [I7HGY0] Fix the issue that the loss of the functional programming does not converge in the PyNative data_sink mode.
• [I7J4N3] Fix the issue that the generation of Step Trace failed in Profiler dynamic Shape mode
• [I7J4N3] Fix the issue that there is no data displayed in the MindInsight parallel strategy view.
• [I79YY4] Fix SiLU operator error when high-order differential in PyNative mode.
• [I6NQJQ] Fix the issue of probabilistic failure in dynamic shape scenarios of the ScatterUpdate operator in PyNative mode.
• [I6Y4G5] Fix the issue of failure in dynamic Shape scenarios of the Conv3D operator in Graph mode.

MindSpore 2.0.0 Release Notes

Major Features and Improvements

PyNative

• [STABLE] Dynamic shape is fully supported on framework. For detailed operator support, refer to Dynamic Shape Support Status of nn Interface, Dynamic Shape Support Status of ops Interface, and Dynamic Shape Support Status of primitive Interface.

AutoParallel

• [STABLE] Build new MindFormers independent repositpry, providing distributed parallel suite, replacing mindspore.nn.transformer module.
• [DEMO] Distributed parallel operator Gather supports the BatchDim attribute.
• [DEMO] Streamline parallel supports specifying any dimension of the input data as the Batch dimension.

API Change

operator

• Add operator primitive for mindspore.ops.AdaptiveAvgPool2D .
• Add operator primitive for mindspore.ops.BatchToSpaceNDV2 .
• Add operator primitive for mindspore.ops.CeLU .
• Add operator primitive for mindspore.ops.ExtractVolumePatches .
• Add operator primitive for mindspore.ops.FFTWithSize .
• Add operator primitive for mindspore.ops.FillDiagonal .
• Add operator primitive for mindspore.ops.FractionalMaxPool3DWithFixedKsize .
• Add operator primitive for mindspore.ops.Im2Col .
• Add operator primitive for mindspore.ops.MaskedScatter .
• Add operator primitive for mindspore.ops.MatrixBandPart .
• Add operator primitive for mindspore.ops.MatrixInverse .
• Add operator primitive for mindspore.ops.MaxPoolWithArgmaxV2 .
• Add operator primitive for mindspore.ops.Ormqr .
• Add operator primitive for mindspore.ops.RandpermV2 .
• Add operator primitive for mindspore.ops.ResizeBicubic .
• Add operator primitive for mindspore.ops.Triu .
• Add operator primitive for mindspore.ops.Zeta .

Backwards Incompatible Change

• Interface: mindspore.ops.MultitypeFuncGraph

  Change: The interface parameter doc_url is used as a test feature in MindSpore 2.0.0.rc1 version. After the optimization of MindSpore 2.0.0 version, users do not need to configure this parameter, so this parameter is deleted in MindSpore 2.0.0 version.

  Original Interface	Interface v2.0.0
  mindspore.ops.MultitypeFuncGraph（name, read_value=False, doc_url=""）	mindspore.ops.MultitypeFuncGraph（name, read_value=False）

• Interface: mindspore.set_context(auto_tune_mode="GA,RL")

  Change: The AutoTune tool has been deprecated, delete auto_tune_mode option, new tuning tools will be planned in the future.

• Interface: mindspore.set_context(mode=PYNATIVE_MODE)

  Change: The default value is changed from GRAPH_MODE to PYNATIVE_MODE.

  Description: If the running mode is not set and the diagram mode needs to be set, use the following method:
  mindspore.set_context(mode=GRAPH_MODE).

  Original Interface	Interface v2.0.0-rc1
  mindspore.set_context(mode=GRAPH_MODE)	mindspore.set_context(mode=PYNATIVE_MODE)

• Interface: mindspore.train.Model.train

  Change: The default value of dataset_sink_mode is changed from True to False.

  Description: If dataset_sink_mode is not set and the data sinking mode needs to be set, use the following method:
  Model.train(dataset_sink_mode=True).

  Original Interface	Interface v2.0.0-rc1
  Model.train(dataset_sink_mode=True)	Model.train(dataset_sink_mode=False)

• Interface: mindspore.export

  Change: The file_format parameter is changed from AIR to no default value.

  Description: If file_format is not set in the original mode, you need to set file_format additionally. In this case, use the following method:
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs).

  Original Interface	Interface v2.0.0-rc1
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs)	mindspore.export(net, *inputs, file_name, file_format, **kwargs)

• Interface: mindspore.ops.norm

  Change: The ord parameter function is extended to support multiple forms.

  Original Interface	Interface v2.0.0-rc1
  ops.norm(input_x, axis, p=2, keep_dims=False, epsilon=1e-12) >>> # Example: >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]], ... [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32)) >>> output = ops.norm(input, [0, 1], p=2)	ops.norm(A, ord=None, dim=None, keepdim=False, *, dtype=None) >>> # Example: >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]], ... [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32)) >>> output = ops.norm(input, ord=2, dim=(0, 1))

• Interface: mindspore.Tensor.norm

  Change: The ord parameter function is extended to support multiple forms.

  Description: For details, see the example of ops.norm.

  Original Interface	Interface v2.0.0-rc1
  Tensor.norm(axis, p=2, keep_dims=False, epsilon=1e-12)	Tensor.norm(ord=None, dim=None, keepdim=False, *, dtype=None)

• Interface: mindspore.ops.dropout

  Change: The seed0 and seed1 parameters are deleted and seed=None parameter is added. Instead of returning Tensors and masks, only Tensors are returned. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout(x, p=0.5, seed0=0, seed1=0) >>> # Example: >>> input = Tensor(((20, 16), (50, 50)), ... mindspore.float32) >>> output, mask = dropout(x, p=0.5)	ops.dropout(input, p=0.5, training=True, seed=None) >>> # Example: >>> input = Tensor(((20, 16), (50, 50)), ... mindspore.float32) >>> output = ops.dropout(input, p=0.5，training=True)

• Interface: mindspore.ops.dropout2d

  Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout2d(x, p=0.5) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output, mask = dropout2d(input, 0.5)	ops.dropout2d(input, p=0.5, training=True) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output = ops.dropout2d(input, 0.5, training=True)

• Interface: mindspore.ops.dropout3d

  Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout3d(x, p=0.5) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output, mask = dropout3d(input, 0.5)	ops.dropout3d(input, p=0.5, training=True) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output = ops.dropout3d(input, 0.5, training=True)

• Interface: mindspore.ops.std

  Change: The interface is reconstructed, and the interface usage mode is more consistent with user habits.

  Description: If parameter unbiased has been set, use the following alternative: unbiased=False -> ddof=0, unbiased=True -> ddof=1.

  Original Interface	Interface v2.0.0-rc1
  ops.std(input_x, axis=(), unbiased=True, keep_dims=False)	ops.std(input, axis=None, ddof=0, keepdims=False)

• Interface: mindspore.load_param_into_net

  Change: Parameters that are not loaded in the ckpt are added as return values.

  Original Interface	Interface v2.0.0-rc1
  net_param = load_param_into_net()	net_param, ckpt_param = load_param_into_net()

• Interface: mindspore.nn.BCELoss

  Change: The default value of reduction is changed from 'none' to 'mean'.

  Original Interface

  Interface v2.0.0-rc1

  BCELoss(weight=None, reduction='none') >>> # Example: >>> weight = Tensor(np.array([[1.0, 2.0, 3.0], ... [4.0, 3.3, 2.2]]), ... mindspore.float32) >>> loss = nn.BCELoss(weight=weight, reduction='mean') >>> logits = Tensor(np.array([[0.1, 0.2, 0.3], ... [0.5, 0.7, 0.9]]), ... mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), ... mindspore.float32) >>> output = loss(logits, labels) >>> print(output) >>> 1.8952923

  BCELoss(weight=None, reduction='mean') >>> # Example: >>> weight = Tensor(np.array([[1.0, 2.0, 3.0], ... [4.0, 3.3, 2.2]]), ... mindspore.float32) >>> loss = nn.BCELoss(weight=weight) >>> logits = Tensor(np.array([[0.1, 0.2, 0.3], ... [0.5, 0.7, 0.9]]), ... mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), ... mindspore.float32) >>> output = loss(logits, labels) >>> print(output) >>> 1.8952923

• Interface: mindspore.ops.split

  Change: The interface is reconstructed. The interface usage mode is more suitable for users. The sequence of the second and third parameters is adjusted, and the split_size_or_sections function is modified and extended.

  Original Interface	Interface v2.0.0-rc1
  ops.split(input_x, axis=0, output_num=1) >>> # Example: >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), ... mindspore.int32) >>> output = ops.split(input, axis=1, output_num=4)	ops.split(tensor, split_size_or_sections, axis=0) >>> # Example: >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), ... mindspore.int32) >>> output = ops.split(input, split_size_or_sections=1, axis=1)

• Interface: mindspore.Tensor.split

  Change: The interface is reconstructed. The interface usage mode is more suitable for users. The positions of the two parameters is adjusted, and the split_size_or_sections function is modified and extended.

  Description: For details, see the example of ops.split.

  Original Interface	Interface v2.0.0-rc1
  Tensor.split(axis=0, output_num=1)	Tensor.split(split_size_or_sections, axis=0)

• Interface: mindspore.ops.pad

  Change: Modify the parameter name paddings to padding, and the mode and value functions are added.

  Original Interface	Interface v2.0.0-rc1
  ops.pad(input_x, paddings) >>> # Example: >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], ... [0.4, 0.5, -3.2]]), ... mindspore.float32) >>> paddings = ((1, 2), (2, 1)) >>> output = ops.pad(input_x, paddings)	ops.pad(input_x, padding, mode='constant', value=None) >>> # Example: >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], ... [0.4, 0.5, -3.2]]), ... mindspore.float32) >>> paddings = (2, 1, 1, 2) >>> output = ops.pad(input_x, paddings)

• Interface: mindspore.ops.meshgrid

  Change: The input parameter is changed from inputs to *input.

  Original Interface	Interface v2.0.0-rc1
  ops.meshgrid(inputs, indexing='xy') >>> # Example: >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) output = ops.meshgrid((x, y, z), indexing='xy')	ops.meshgrid(*inputs, indexing='xy') >>> # Example: >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) output = ops.meshgrid(x, y, z, indexing='xy')

• Interface: mindspore.ops.max

  Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

  Original Interface	Interface v2.0.0-rc1
  ops.max(x, axis=0, keep_dims=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> index, output = ops.max(input) >>> print(index, output) >>> 3 0.7	ops.max(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> output, index = ops.max(input, axis=0) >>> print(output, index)

• Interface: mindspore.ops.min

  Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

  Original Interface	Interface v2.0.0-rc1
  ops.min(x, axis=0, keep_dims=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> index, output = ops.min(input) >>> 0 0.0	ops.min(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> output, index = ops.min(input, keepdims=True) >>> 0.0 0

• Interface: mindspore.ops.random_gamma

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.random_gamma(shape, alpha, seed=0, seed2=0)	ops.random_gamma(shape, alpha, seed=None)

• Interface: mindspore.ops.standard_laplace

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.standard_laplace(shape, seed=0, seed2=0)	ops.standard_laplace(shape, seed=None)

• Interface: mindspore.ops.standard_normal

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.standard_normal(shape, seed=0, seed2=0)	ops.standard_normal(shape, seed=None)

• Interface: mindspore.ops.bernoulli

  Change: The default value of seed is changed from -1 to None. Meets the actual application scenario.

  Original Interface	Interface v2.0.0-rc1
  ops.bernoulli(x, p=0.5, seed=-1)	ops.bernoulli(input, p=0.5, seed=None)

• Interface: mindspore.data_sink

  Change: Deleted the steps parameter. Parameter name jit is changed to jit_config, and new input_signature parameter is added. The usability is improved to meet the requirements of actual application scenarios.

  Original Interface	Interface v2.0.0-rc1
  mindspore.data_sink(fn, dataset, steps, sink_size=1, jit=False)	mindspore.data_sink(fn, dataset, sink_size=1, jit_config=None, input_signature=None)

• Interface: mindspore.ops.conv2d

  Change: Extend Interface Function. Add the bias parameter and modify the parameter name and parameter sequence.

  Original Interface	Interface v2.0.0-rc1
  conv2d(inputs, weight, pad_mode="valid", padding=0, stride=1, dilation=1, group=1)	conv2d(input, weight, bias=None, stride=1, pad_mode="valid", padding=0, dilation=1, groups=1)

• Interface: mindspore.dataset.vision.Pad

  Change: Adjust the input parameter padding of Pad, RandomCrop, and RandomCropWithBbox. When the input length of Padding is 2, the first value is used to fill the left/upper boundary, the second value is used to fill the right/lower boundary, and the first value is used to fill the left/right boundary. Fill the upper/lower boundary with the second value.

  Description: The padding parameter whose size is 2 is not compatible with the effect of the earlier version. The padding parameter needs to be explicitly represented (left, right, top, and bottom).

  Original Interface	Interface v2.0.0-rc1
  mindspore.dataset.vision.Pad(padding=(1,2)) Indicates that the left/upper part of the image is filled with 1 pixel, and the right/down part is filled with 2 pixels.	mindspore.dataset.vision.Pad(padding=(1,2,1,2)) Indicates that the left/upper part of the image is filled with 1 pixel, and the right/down part is filled with 2 pixels.

• Interface: mindspore.dataset.Dataset.map

  Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

  Description:

  1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.map(operations=[transforms], ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"], ... column_order=["column_c", "column_b"])	>>> dataset = dataset.map(operations=[transforms], ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"]) >>> dataset = dataset.project(["column_c", column_b"])")

• Interface: mindspore.dataset.Dataset.batch

  Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

  Description:

  1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.batch(batch_size=4, ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"], ... column_order=["column_c", "column_b"])	>>> dataset = dataset.batch(batch_size=4, input_columns=["column_a"] ... output_columns=["column_b", "column_c"]) >>> dataset = dataset.project(["column_c", column_b"])")

• Interface: mindspore.dataset.Dataset.batch

  Change: Split the batch method into two methods: batch and padded_batch. The pad_info parameter is moved from the batch method to the padded_batch method.

  Description: To use the pad_info parameter, use the padded_batch method instead.

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.batch(batch_size=4, ... drop_remainder=True, pad_info=...)	>>> dataset = dataset.padded_batch(batch_size=4, ... drop_remainder=True, pad_info=...)

Bug fixes

• [I62I3J] fix inference failure of BGCF network on Ascend 310
• [I7C2W3] fix error issuse of null pointer when enabling multiple loss in parallel pipeline scenarios

Contributors

Thanks goes to these wonderful people:

alashkari,anzhengqi,archer2049,B.L.LAN,baihuawei,bichaoyang,BJ-WANG,Bokai Li,Brian-K,caifubi,caiyimeng,cathwong,changzherui,ChenDonYY,chenfei_mindspore,chengang,chengbin,chenhaozhe,chenjianping,chenkang,chenweifeng,chuht,chujinjin,davidanugraha,DavidFFFan,DeshiChen,douzhixing,emmmmtang,Erpim,Ethan,fangwenyi,fangzehua,fangzhou0329,fary86,fengyixing,gaoshuanglong,Gaoxiong,gaoyong10,gengdongjie,gongdaguo1,Greatpan,GuoZhibin,guozhijian,hangq,hanhuifeng,haozhang,hedongdong,Henry Shi,heterogeneous_to_backoff_2_0,huangbingjian,huanghui,huangxinjing,hujiahui8,hujingsong,huoxinyou,jachua,jiahongQian,jianghui58,jiangzhenguang,jiaorui,jiaoy1224,jijiarong,jjfeing,JoeyLin,json,JuiceZ,jxl,kairui_kou,KevinYi,kisnwang,KXiong,laiyongqiang,lanzhineng,liangchenghui,liangzelang,LiangZhibo,lianliguang,lichen,ligan,lijunbin,limingqi107,ling,linqingke,liubuyu,liuchao,liuchuting,liujunzhu,liuluobin,liutongtong9,liuyang811,lixiao,liyan2022,liyejun,liyuxia,looop5,luochao60,luojianing,luoyang,luoyuan,lyqlola,maning202007,maoyaomin,Margaret_wangrui,mayadong,MaZhiming,melody,mengyuanli,michaelzhu_70ab,Mohammad Motallebi,moran,NaCN,nomindcarry,OwenSec,panfengfeng,panshaowu,panzhihui,pkuliuliu,qinzheng,qiuzhongya,qujianwei,r1chardf1d0,Renyuan Zhang,RobinGrosman,shaojunsong,shenwei41,Soaringfish,tangdezhi_123,tanghuikang,tan-wei-cheng,TinaMengtingZhang,TronZhang,TuDouNi,VectorSL,wang_ziqi,wanghenchang,wangnan39,wangpingan,wangshaocong,wangshengnan123,wangtongyu6,weichaoran,wind-zyx,wqx,wtcheng,wujueying,wYann,XianglongZeng,xiaohanzhang,xiaotianci,xiaoyao,XinDu,xulei,xumengjuan1,xupan,xwkgch,yanghaoran,yangluhang,yangruoqi713,yangshuo,yangsijia,yangzhenzhang,yanzhenxiang2020,Yanzhi_YI,yao_yf,yefeng,yeyunpeng2020,Yi_zhang95,yide12,YijieChen,YingLai Lin,YingtongHu,youshu,yuchaojie,yuedongli,YuJianfeng,zangqx,ZengZitao,zhangbuxue,zhangdanyang,zhangdong,zhangfanghe,zhangqi,zhangqinghua,zhangyanhui,zhangyinxia,zhangyongxian,zhangzhaoju,zhanzhan,zhengzuohe,ZhidanLiu,zhixinaa,zhoufeng,zhouyaqiang0,zhuguodong,zhupuxu,zhuyuxiao,zichun_ye,zjun,zlq2020,zong_shuai,ZPaC,zuochuanyong,zyli2020,陈宇,范吉斌,冯一航,胡彬,宦晓玲,黄勇,雷元哲,李良灿,李林杰,刘崇鸣,刘力力,刘勇琪,吕浩宇,吕昱峰（Nate.River）,没有窗户的小巷,沈竞兴,十六夜,王程浩,王禹程,王振邦,徐安越,徐永飞,杨旭华,于振华,俞涵,张清华,张澍坤,张栩浩,张学同,赵英灼,周超,周洪叶,朱家兴

Contributions of any kind are welcome!

MindSpore 2.0.0-rc1 Release Notes

Major Features and Improvements

FrontEnd

• [BETA] Statement with "return", "return None" and with no return of function are supported in GRAPH_MODE.
• [BETA] Object with list type are supported in GRAPH_MODE.
• [BETA] Statement with "raise" are supported in variable condition situation in GRAPH_MODE.
• [STABLE] Functional call supports data sinking mode.
• [BETA] The Transformer layer in nn module is added to provide easy-to-use Transformer APIs. Batch_size does not need to be defined. Dynamic seq_length is supported.

DataSet

• [STABLE] In the Ascend environment，the timeout waiting time in data sink mode is adjusted to 1900s by default. This solves the problem that the GetNext operator may time out due to environment resource competition and large computing workload in data sinking mode.
• [STABLE] MindRecord supports to query the schemas and number samples. MindRecord provides multi-process writing mode, allowing users to generate MindRecord data files in parallel.
• [STABLE] The Dataset pipeline can process any Python object. For details, see Supporting Python Objects in Dataset Pipeline.

AutoParallel

• [STABLE] The strategies of whole parameters can be saved when saving strategy.
• [STABLE] The Conv3D/MaxPool3D/AvgPool3D distributed operator is supported.
• [STABLE] Support operator-level parallelism and optimizer-level parallelism under the PyNative with shard: parallel training and the Model API are decoupled to provide basic parallel expression capabilities.
• [STABLE] Support operator-level parallelism, and optimizer-level parallelism under the Graph mode: parallel training and the Model API are decoupled to provide basic parallel expression capabilities.
• [BETA] Supports customized distributed graph segmentation, improving the flexibility of distributed training.

Runtime

• [STABLE] Control flow supports subgraph sink.
• [STABLE] Support CUDA 11.6.
• [STABLE] Support for operator selection and execution of List/Tuple/Scalar type kernel to match native Python expression.
• [STABLE] Kernel that is not supported by hardware can automatically select CPU kernel.
• [STABLE] Support heterogeneous execution within subgraph.

Ascend

• [STABLE] Support overflow detection scheme and HCCL runtime overflow check.
• [STABLE] Support dump of communication operators.

Profiler

• [STABLE] Rich Profiler collection item configuration, users can collect performance data in more detail.

Dump

• [BETA] Single card in PyNatvie mode supports operator overflow detection.
• [BETA] Graph mode supports hccl operator dump.

API Change

• [STABLE] Add computing APIs, such as MaxUnpool, ReplicationPad, and GaussianNLLLoss.
  For details, visit https://www.mindspore.cn/docs/en/r2.0/api_python/mindspore.html.
• [STABLE] Extend inventory API functions, such as AvgPool, pad, norm, and interplate.

operator

• [BETA] Add operator primitive for mindspore.ops.AdaptiveAvgPool3D.
• [BETA] Add operator primitive for mindspore.ops.AffineGrid.
• [BETA] Add operator primitive for mindspore.ops.Angle.
• [BETA] Add operator primitive for mindspore.ops.BartlettWindow.
• [BETA] Add operator primitive for mindspore.ops.Bernoulli.
• [BETA] Add operator primitive for mindspore.ops.BesselI0.
• [BETA] Add operator primitive for mindspore.ops.BesselI1.
• [BETA] Add operator primitive for mindspore.ops.BesselJ0.
• [BETA] Add operator primitive for mindspore.ops.BesselJ1.
• [BETA] Add operator primitive for mindspore.ops.BesselK0.
• [BETA] Add operator primitive for mindspore.ops.BesselK0e.
• [BETA] Add operator primitive for mindspore.ops.BesselK1.
• [BETA] Add operator primitive for mindspore.ops.BesselK1e.
• [BETA] Add operator primitive for mindspore.ops.BesselY0.
• [BETA] Add operator primitive for mindspore.ops.BesselY1.
• [BETA] Add operator primitive for mindspore.ops.Bincount.
• [BETA] Add operator primitive for mindspore.ops.BlackmanWindow.
• [BETA] Add operator primitive for mindspore.ops.ChannelShuffle.
• [BETA] Add operator primitive for mindspore.ops.Cholesky.
• [BETA] Add operator primitive for mindspore.ops.Col2Im.
• [BETA] Add operator primitive for mindspore.ops.Complex.
• [BETA] Add operator primitive for mindspore.ops.ComplexAbs.
• [BETA] Add operator primitive for mindspore.ops.Cross.
• [BETA] Add operator primitive for mindspore.ops.CTCLossV2.
• [BETA] Add operator primitive for mindspore.ops.Cummin.
• [BETA] Add operator primitive for mindspore.ops.Diag.
• [BETA] Add operator primitive for mindspore.ops.Digamma.
• [BETA] Add operator primitive for mindspore.ops.Expand.
• [BETA] Add operator primitive for mindspore.ops.Fmax.
• [BETA] Add operator primitive for mindspore.ops.Gcd.
• [BETA] Add operator primitive for mindspore.ops.Geqrf.
• [BETA] Add operator primitive for mindspore.ops.GLU.
• [BETA] Add operator primitive for mindspore.ops.GridSampler2D.
• [BETA] Add operator primitive for mindspore.ops.GridSampler3D.
• [BETA] Add operator primitive for mindspore.ops.HammingWindow.
• [BETA] Add operator primitive for mindspore.ops.Heaviside.
• [BETA] Add operator primitive for mindspore.ops.Hypot.
• [BETA] Add operator primitive for mindspore.ops.Igamma.
• [BETA] Add operator primitive for mindspore.ops.IndexFill.
• [BETA] Add operator primitive for mindspore.ops.InplaceIndexAdd.
• [BETA] Add operator primitive for mindspore.ops.InplaceUpdateV2.
• [BETA] Add operator primitive for mindspore.ops.Lcm.
• [BETA] Add operator primitive for mindspore.ops.LeftShift.
• [BETA] Add operator primitive for mindspore.ops.LogicalXor.
• [BETA] Add operator primitive for mindspore.ops.Logit.
• [BETA] Add operator primitive for mindspore.ops.LogSpace.
• [BETA] Add operator primitive for mindspore.ops.LuUnpack.
• [BETA] Add operator primitive for mindspore.ops.MatrixDiagPartV3.
• [BETA] Add operator primitive for mindspore.ops.MatrixDiagV3.
• [BETA] Add operator primitive for mindspore.ops.MatrixSetDiagV3.
• [BETA] Add operator primitive for mindspore.ops.MaxPool3DWithArgmax.
• [BETA] Add operator primitive for mindspore.ops.MaxUnpool2D.
• [BETA] Add operator primitive for mindspore.ops.MaxUnpool3D.
• [BETA] Add operator primitive for mindspore.ops.MultiMarginLoss.
• [BETA] Add operator primitive for mindspore.ops.MultinomialWithReplacement.
• [BETA] Add operator primitive for mindspore.ops.Mvlgamma.
• [BETA] Add operator primitive for mindspore.ops.NanToNum.
• [BETA] Add operator primitive for mindspore.ops.NextAfter.
• [BETA] Add operator primitive for mindspore.ops.Orgqr.
• [BETA] Add operator primitive for mindspore.ops.Polygamma.
• [BETA] Add operator primitive for mindspore.ops.ResizeBilinearV2.
• [BETA] Add operator primitive for mindspore.ops.RightShift.
• [BETA] Add operator primitive for mindspore.ops.ScatterNdDiv.
• [BETA] Add operator primitive for mindspore.ops.ScatterNdMul.
• [BETA] Add operator primitive for mindspore.ops.SearchSorted.
• [BETA] Add operator primitive for mindspore.ops.Sinc.
• [BETA] Add operator primitive for mindspore.ops.Trace.
• [BETA] Add operator primitive for mindspore.ops.Tril.
• [BETA] Add operator primitive for mindspore.ops.TrilIndices.
• [BETA] Add operator primitive for mindspore.ops.TriuIndices.
• [BETA] Add operator primitive for mindspore.ops.UniqueConsecutive.
• [STABLE] Add operator primitive for mindspore.ops.Cummax.
• [STABLE] Add operator primitive for mindspore.ops.FillV2.
• [STABLE] Add operator primitive for mindspore.ops.IsClose.
• [STABLE] Add operator primitive for mindspore.ops.MatrixSolve.
• [STABLE] Add operator primitive for mindspore.ops.Median.
• [STABLE] Add operator primitive for mindspore.ops.MultilabelMarginLoss.
• [STABLE] Add operator primitive for mindspore.ops.NonZero.
• [STABLE] Add operator primitive for mindspore.ops.Pdist.
• [STABLE] Add operator primitive for mindspore.ops.Polar.
• [STABLE] Add operator primitive for mindspore.ops.RandomGamma.
• [STABLE] Add operator primitive for mindspore.ops.RandomPoisson.
• [STABLE] Add operator primitive for mindspore.ops.RandomShuffle.
• [STABLE] Add operator primitive for mindspore.ops.Renorm.
• [STABLE] Add operator primitive for mindspore.ops.ScatterNdMax.
• [STABLE] Add operator primitive for mindspore.ops.ScatterNdMin.
• [STABLE] Add operator primitive for mindspore.ops.Svd.
• [STABLE] Add operator primitive for mindspore.ops.TripletMarginLoss.

Deleted APIs

• The mindspore.compression feature was deprecated at MindSpore 1.8 and is removed in this version.
  The following mindspore.nn.quant interfaces are also removed simultaneously: mindspore.nn.FakeQuantWithMinMaxObserver, mindspore.nn.Conv2dBnFoldQuantOneConv, mindspore.nn.Conv2dBnFoldQuant, mindspore.nn.Conv2dBnWithoutFoldQuant, mindspore.nn.Conv2dQuant, mindspore.nn.DenseQuant, mindspore.nn.ActQuant, mindspore.nn.TensorAddQuant, mindspore.nn.ActQuant, mindspore.nn.MulQuant. Please use MindSpore Golden Stick instead to implement QuantAwareTraining in MindSpore.
• The mindspore.dataset.close_pool, mindspore.dataset.to_device, and mindspore.dataset.set_dynamic_columns interfaces are discarded in earlier version and being removed in this version.

Backwards Incompatible Change

• Interface: mindspore.set_context(mode=PYNATIVE_MODE)

  Change: The default value is changed from GRAPH_MODE to PYNATIVE_MODE.

  Description: If the running mode is not set and the diagram mode needs to be set, use the following method:
  mindspore.set_context(mode=GRAPH_MODE).

  Original Interface	Interface v2.0.0-rc1
  mindspore.set_context(mode=GRAPH_MODE)	mindspore.set_context(mode=PYNATIVE_MODE)

• Interface: mindspore.train.Model.train

  Change: The default value of dataset_sink_mode is changed from True to False.

  Description: If dataset_sink_mode is not set and the data sinking mode needs to be set, use the following method:
  Model.train(dataset_sink_mode=True).

  Original Interface	Interface v2.0.0-rc1
  Model.train(dataset_sink_mode=True)	Model.train(dataset_sink_mode=False)

• Interface: mindspore.export

  Change: The file_format parameter is changed from AIR to no default value.

  Description: If file_format is not set in the original mode, you need to set file_format additionally. In this case, use the following method:
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs).

  Original Interface	Interface v2.0.0-rc1
  mindspore.export(net, *inputs, file_name, file_format="AIR", **kwargs)	mindspore.export(net, *inputs, file_name, file_format, **kwargs)

• Interface: mindspore.ops.norm

  Change: The ord parameter function is extended to support multiple forms.

  Original Interface	Interface v2.0.0-rc1
  ops.norm(input_x, axis, p=2, keep_dims=False, epsilon=1e-12) >>> # Example: >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]], ... [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32)) >>> output = ops.norm(input, [0, 1], p=2)	ops.norm(A, ord=None, dim=None, keepdim=False, *, dtype=None) >>> # Example: >>> input = Tensor(np.array([[[1.0, 2.0], [3.0, 4.0]], ... [[5.0, 6.0], [7.0, 8.0]]]).astype(np.float32)) >>> output = ops.norm(input, ord=2, dim=(0, 1))

• Interface: mindspore.Tensor.norm

  Change: The ord parameter function is extended to support multiple forms.

  Description: For details, see the example of ops.norm.

  Original Interface	Interface v2.0.0-rc1
  Tensor.norm(axis, p=2, keep_dims=False, epsilon=1e-12)	Tensor.norm(ord=None, dim=None, keepdim=False, *, dtype=None)

• Interface: mindspore.ops.dropout

  Change: The seed0 and seed1 parameters are deleted and seed=None parameter is added. Instead of returning Tensors and masks, only Tensors are returned. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout(x, p=0.5, seed0=0, seed1=0) >>> # Example: >>> input = Tensor(((20, 16), (50, 50)), ... mindspore.float32) >>> output, mask = dropout(x, p=0.5)	ops.dropout(input, p=0.5, training=True, seed=None) >>> # Example: >>> input = Tensor(((20, 16), (50, 50)), ... mindspore.float32) >>> output = ops.dropout(input, p=0.5，training=True)

• Interface: mindspore.ops.dropout2d

  Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout2d(x, p=0.5) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output, mask = dropout2d(input, 0.5)	ops.dropout2d(input, p=0.5, training=True) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output = ops.dropout2d(input, 0.5, training=True)

• Interface: mindspore.ops.dropout3d

  Change: Return value is changed from Tensor and mask to Tensor only. The input parameter training=True is added.

  Original Interface	Interface v2.0.0-rc1
  ops.dropout3d(x, p=0.5) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output, mask = dropout3d(input, 0.5)	ops.dropout3d(input, p=0.5, training=True) >>> # Example: >>> input = Tensor(np.ones([2, 1, 2, 3]), ... mindspore.float32) >>> output = ops.dropout3d(input, 0.5, training=True)

• Interface: mindspore.ops.std

  Change: The interface is reconstructed, and the interface usage mode is more consistent with user habits.

  Description: If parameter unbiased has been set, use the following alternative: unbiased=False -> ddof=0, unbiased=True -> ddof=1.

  Original Interface	Interface v2.0.0-rc1
  ops.std(input_x, axis=(), unbiased=True, keep_dims=False)	ops.std(input, axis=None, ddof=0, keepdims=False)

• Interface: mindspore.load_param_into_net

  Change: Parameters that are not loaded in the ckpt are added as return values.

  Original Interface	Interface v2.0.0-rc1
  net_param = load_param_into_net()	net_param, ckpt_param = load_param_into_net()

• Interface: mindspore.nn.BCELoss

  Change: The default value of reduction is changed from 'none' to 'mean'.

  Original Interface

  Interface v2.0.0-rc1

  BCELoss(weight=None, reduction='none') >>> # Example: >>> weight = Tensor(np.array([[1.0, 2.0, 3.0], ... [4.0, 3.3, 2.2]]), ... mindspore.float32) >>> loss = nn.BCELoss(weight=weight, reduction='mean') >>> logits = Tensor(np.array([[0.1, 0.2, 0.3], ... [0.5, 0.7, 0.9]]), ... mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), ... mindspore.float32) >>> output = loss(logits, labels) >>> print(output) >>> 1.8952923

  BCELoss(weight=None, reduction='mean') >>> # Example: >>> weight = Tensor(np.array([[1.0, 2.0, 3.0], ... [4.0, 3.3, 2.2]]), ... mindspore.float32) >>> loss = nn.BCELoss(weight=weight) >>> logits = Tensor(np.array([[0.1, 0.2, 0.3], ... [0.5, 0.7, 0.9]]), ... mindspore.float32) >>> labels = Tensor(np.array([[0, 1, 0], [0, 0, 1]]), ... mindspore.float32) >>> output = loss(logits, labels) >>> print(output) >>> 1.8952923

• Interface: mindspore.ops.split

  Change: The interface is reconstructed. The interface usage mode is more suitable for users. The sequence of the second and third parameters is adjusted, and the split_size_or_sections function is modified and extended.

  Original Interface	Interface v2.0.0-rc1
  ops.split(input_x, axis=0, output_num=1) >>> # Example: >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), ... mindspore.int32) >>> output = ops.split(input, axis=1, output_num=4)	ops.split(tensor, split_size_or_sections, axis=0) >>> # Example: >>> input = Tensor(np.array([[1, 1, 1, 1], [2, 2, 2, 2]]), ... mindspore.int32) >>> output = ops.split(input, split_size_or_sections=1, axis=1)

• Interface: mindspore.Tensor.split

  Change: The interface is reconstructed. The interface usage mode is more suitable for users. The positions of the two parameters is adjusted, and the split_size_or_sections function is modified and extended.

  Description: For details, see the example of ops.split.

  Original Interface	Interface v2.0.0-rc1
  Tensor.split(axis=0, output_num=1)	Tensor.split(split_size_or_sections, axis=0)

• Interface: mindspore.ops.pad

  Change: Modify the parameter name paddings to padding, and the mode and value functions are added.

  Original Interface	Interface v2.0.0-rc1
  ops.pad(input_x, paddings) >>> # Example: >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], ... [0.4, 0.5, -3.2]]), ... mindspore.float32) >>> paddings = ((1, 2), (2, 1)) >>> output = ops.pad(input_x, paddings)	ops.pad(input_x, padding, mode='constant', value=None) >>> # Example: >>> input_x = Tensor(np.array([[-0.1, 0.3, 3.6], ... [0.4, 0.5, -3.2]]), ... mindspore.float32) >>> paddings = (2, 1, 1, 2) >>> output = ops.pad(input_x, paddings)

• Interface: mindspore.ops.meshgrid

  Change: The input parameter is changed from inputs to *input.

  Original Interface	Interface v2.0.0-rc1
  ops.meshgrid(inputs, indexing='xy') >>> # Example: >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) output = ops.meshgrid((x, y, z), indexing='xy')	ops.meshgrid(*inputs, indexing='xy') >>> # Example: >>> x = Tensor(np.array([1, 2, 3, 4]).astype(np.int32)) >>> y = Tensor(np.array([5, 6, 7]).astype(np.int32)) >>> z = Tensor(np.array([8, 9, 0, 1, 2]).astype(np.int32)) output = ops.meshgrid(x, y, z, indexing='xy')

• Interface: mindspore.ops.max

  Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

  Original Interface	Interface v2.0.0-rc1
  ops.max(x, axis=0, keep_dims=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> index, output = ops.max(input) >>> print(index, output) >>> 3 0.7	ops.max(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> output, index = ops.max(input, axis=0) >>> print(output, index)

• Interface: mindspore.ops.min

  Change: Return value exchange sequence. The value is changed from "index, value" to "value, index".

  Original Interface	Interface v2.0.0-rc1
  ops.min(x, axis=0, keep_dims=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> index, output = ops.min(input) >>> 0 0.0	ops.min(input, axis=None, keepdims=False, *, initial=None, where=True, return_indices=False) >>> # Example: >>> input = Tensor(np.array([0.0, 0.4, 0.6, 0.7, 0.1]), ... mindspore.float32) >>> output, index = ops.min(input, keepdims=True) >>> 0.0 0

• Interface: mindspore.ops.random_gamma

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.random_gamma(shape, alpha, seed=0, seed2=0)	ops.random_gamma(shape, alpha, seed=None)

• Interface: mindspore.ops.standard_laplace

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.standard_laplace(shape, seed=0, seed2=0)	ops.standard_laplace(shape, seed=None)

• Interface: mindspore.ops.standard_normal

  Change: The seed2 parameter is deleted and seed=0 is changed to None. The framework behavior is unified and complies with the actual application scenarios and habits of users.

  Original Interface	Interface v2.0.0-rc1
  ops.standard_normal(shape, seed=0, seed2=0)	ops.standard_normal(shape, seed=None)

• Interface: mindspore.ops.bernoulli

  Change: The default value of seed is changed from -1 to None. Meets the actual application scenario.

  Original Interface	Interface v2.0.0-rc1
  ops.bernoulli(x, p=0.5, seed=-1)	ops.bernoulli(input, p=0.5, seed=None)

• Interface: mindspore.data_sink

  Change: Deleted the steps parameter. Parameter name jit is changed to jit_config, and new input_signature parameter is added. The usability is improved to meet the requirements of actual application scenarios.

  Original Interface	Interface v2.0.0-rc1
  mindspore.data_sink(fn, dataset, steps, sink_size=1, jit=False)	mindspore.data_sink(fn, dataset, sink_size=1, jit_config=None, input_signature=None)

• Interface: mindspore.ops.conv2d

  Change: Extend Interface Function. Add the bias parameter and modify the parameter name and parameter sequence.

  Original Interface	Interface v2.0.0-rc1
  conv2d(inputs, weight, pad_mode="valid", padding=0, stride=1, dilation=1, group=1)	conv2d(input, weight, bias=None, stride=1, pad_mode="valid", padding=0, dilation=1, groups=1)

• Interface: mindspore.dataset.vision.Pad

  Change: Adjust the input parameter padding of Pad, RandomCrop, and RandomCropWithBbox. When the input length of Padding is 2, the first value is used to fill the left/upper boundary, the second value is used to fill the right/lower boundary, and the first value is used to fill the left/right boundary. Fill the upper/lower boundary with the second value.

  Description: The padding parameter whose size is 2 is not compatible with the effect of the earlier version. The padding parameter needs to be explicitly represented (left, right, top, and bottom).

  Original Interface	Interface v2.0.0-rc1
  mindspore.dataset.vision.Pad(padding=(1,2)) Indicates that the left/upper part of the image is filled with 1 pixel, and the right/down part is filled with 2 pixels.	mindspore.dataset.vision.Pad(padding=(1,2,1,2)) Indicates that the left/upper part of the image is filled with 1 pixel, and the right/down part is filled with 2 pixels.

• Interface: mindspore.dataset.Dataset.map

  Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

  Description:

  1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.map(operations=[transforms], ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"], ... column_order=["column_c", "column_b"])	>>> dataset = dataset.map(operations=[transforms], ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"]) >>> dataset = dataset.project(["column_c", column_b"])")

• Interface: mindspore.dataset.Dataset.batch

  Change: Delete the column_order parameter. In most cases, output_columns and column_order have the same value. Therefore, column_order does not need to be transferred. To adjust the sequence of data columns, use mindspore.dataset.Dataset.project.

  Description:

  1. If the column sequence does not need to be changed, delete the column_order parameter.
  2. If you need to specify the data column sequence, delete the column_order parameter and add a project method to the end of the parameter for column transformation (as in the following example).

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.batch(batch_size=4, ... input_columns=["column_a"], ... output_columns=["column_b", "column_c"], ... column_order=["column_c", "column_b"])	>>> dataset = dataset.batch(batch_size=4, input_columns=["column_a"] ... output_columns=["column_b", "column_c"]) >>> dataset = dataset.project(["column_c", column_b"])")

• Interface: mindspore.dataset.Dataset.batch

  Change: Split the batch method into two methods: batch and padded_batch. The pad_info parameter is moved from the batch method to the padded_batch method.

  Description: To use the pad_info parameter, use the padded_batch method instead.

  Original Interface	Interface v2.0.0-rc1
  >>> dataset = dataset.batch(batch_size=4, ... drop_remainder=True, pad_info=...)	>>> dataset = dataset.padded_batch(batch_size=4, ... drop_remainder=True, pad_info=...)

Bug fixes

• [I66PE6] fix AssignSub primitive abnormal input leads to coredump.

• [I6F5E6] fix data_sink function timeout on Ascend.

Others

• Windows support is still being optimized,this version does not support now.It will be available for download in version 2.0.

Contributors

Thanks goes to these wonderful people:

alashkari,anzhengqi,archer2049,B.L.LAN,baihuawei,bichaoyang,BJ-WANG,Bokai Li,Brian-K,caifubi,caiyimeng,cathwong,changzherui,ChenDonYY,chenfei_mindspore,chengang,chengbin,chenhaozhe,chenjianping,chenkang,chenweifeng,chuht,chujinjin,davidanugraha,DavidFFFan,DeshiChen,douzhixing,emmmmtang,Erpim,Ethan,fangwenyi,fangzehua,fangzhou0329,fary86,fengyixing,gaoshuanglong,Gaoxiong,gaoyong10,gengdongjie,gongdaguo1,Greatpan,GuoZhibin,guozhijian,hangq,hanhuifeng,haozhang,hedongdong,Henry Shi,heterogeneous_to_backoff_2_0,huangbingjian,huanghui,huangxinjing,hujiahui8,hujingsong,huoxinyou,jachua,jiahongQian,jianghui58,jiangzhenguang,jiaorui,jiaoy1224,jijiarong,jjfeing,JoeyLin,json,JuiceZ,jxl,kairui_kou,KevinYi,kisnwang,KXiong,laiyongqiang,lanzhineng,liangchenghui,liangzelang,LiangZhibo,lianliguang,lichen,ligan,lijunbin,limingqi107,ling,linqingke,liubuyu,liuchao,liuchuting,liujunzhu,liuluobin,liutongtong9,liuyang811,lixiao,liyan2022,liyejun,liyuxia,looop5,luochao60,luojianing,luoyang,luoyuan,lyqlola,maning202007,maoyaomin,Margaret_wangrui,mayadong,MaZhiming,melody,mengyuanli,michaelzhu_70ab,Mohammad Motallebi,moran,NaCN,nomindcarry,OwenSec,panfengfeng,panshaowu,panzhihui,pkuliuliu,qinzheng,qiuzhongya,qujianwei,r1chardf1d0,Renyuan Zhang,RobinGrosman,shaojunsong,shenwei41,Soaringfish,tangdezhi_123,tanghuikang,tan-wei-cheng,TinaMengtingZhang,TronZhang,TuDouNi,VectorSL,wang_ziqi,wanghenchang,wangnan39,wangpingan,wangshaocong,wangshengnan123,wangtongyu6,weichaoran,wind-zyx,wqx,wtcheng,wujueying,wYann,XianglongZeng,xiaohanzhang,xiaotianci,xiaoyao,XinDu,xulei,xumengjuan1,xupan,xwkgch,yanghaoran,yangluhang,yangruoqi713,yangshuo,yangsijia,yangzhenzhang,yanzhenxiang2020,Yanzhi_YI,yao_yf,yefeng,yeyunpeng2020,Yi_zhang95,yide12,YijieChen,YingLai Lin,YingtongHu,youshu,yuchaojie,yuedongli,YuJianfeng,zangqx,ZengZitao,zhangbuxue,zhangdanyang,zhangdong,zhangfanghe,zhangqi,zhangqinghua,zhangyanhui,zhangyinxia,zhangyongxian,zhangzhaoju,zhanzhan,zhengzuohe,ZhidanLiu,zhixinaa,zhoufeng,zhouyaqiang0,zhuguodong,zhupuxu,zhuyuxiao,zichun_ye,zjun,zlq2020,zong_shuai,ZPaC,zuochuanyong,zyli2020,陈宇,范吉斌,冯一航,胡彬,宦晓玲,黄勇,雷元哲,李良灿,李林杰,刘崇鸣,刘力力,刘勇琪,吕浩宇,吕昱峰（Nate.River）,没有窗户的小巷,沈竞兴,十六夜,王程浩,王禹程,王振邦,徐安越,徐永飞,杨旭华,于振华,俞涵,张清华,张澍坤,张栩浩,张学同,赵英灼,周超,周洪叶,朱家兴

Contributions of any kind are welcome!

MindSpore 2.0.0-alpha Release Notes

Major Features and Improvements

PyNative

• The default mode of MindSpore is switched to PyNative. If you want to manually set the mode, please refer to Computational Graph.
• Support dynamic shape without padding, three networks are supported as demos: Transformer-GPU, YOLOV5-GPU, ASR-Ascend. Transformer-GPU and YOLOV5-GPU can be downloaded from models. Only the following operators are available on Ascend backend: Add、Assign、BatchMatMul、BiasAdd、BiasAddGrad、Cast、Conv2D、Conv2DBackpropFilter、Conv2DBackpropInput、CTCLoss、Div、Dropout、DropoutDoMask、Equal、ExpandDims、Gather、GetNext、LayerNorm、LayerNormGrad、LessEqual、Load、Log、LogicalAnd、LogicalNot、LogicalOr、LogSoftmax、LogSoftmaxGrad、MatMul、Maximum、Mul、Neg、NotEqual、NPUAllocFloatStatus、NPUClearFloatStatus、OneHot、RealDiv、Reciprocal、ReduceMean、ReduceSum、ReLU、ReluGrad、Reshape、Select、Softmax、StridedSlice、Sub、Tile、Transpose、UnsortedSegmentSum、ZerosLike。The remaining operators have not been fully verified, please use them as appropriate.

DataSet

• The TFRecordDataset API can directly read TFRecord files compressed by GZIP or ZLIB.
• The NumpySlicesDataset API can process data of different dimensions at the same time.
• Optimize the structure of error log to display more clear call stack information for debugging.
• Fixed mindspore.dataset.config.set_seed does not take effect for random seeds in distributed training scenarios.

AutoParallel

• Supports more operators with distributed implements.

  Element Wise Operators:AddN, BitwiseAnd, BitwiseOr, BitwiseXor, CumProd, HShrink, HSigmoid, IsFinite, Mish, MulNoNan, Rint, SeLU, SoftShrink, TruncateDiv, TruncateMod, Xdivy Xlogy, InplaceAdd, InplacSub, InplaceUpdate, Cdist, L2Loss, Lerp.

  Math Operators:SquaredDifference, Erfinv, MaskedFill, SplitV, Gamma, KLDivLoss, LinSpace.

  Scatter Operators:ScatterAdd,ScatterDiv,ScatterMax,ScatterMul,ScatterNdAdd,ScatterNdSub,ScatterNdUpdate,ScatterSub,TensorScatterAdd,TensorScatterDiv,TensorScatterMax,TensorScatterMax,TensorScatterMul,TensorScatterAdd,TensorScatterUpdate.

• Add new apis transform_checkpoints and transform_checkpoint_by_rank to transfer the distributed checkpoint files by strategy files. Please refer to Distributed Resilience Training and Inference。

API Change

operator

• [STABLE] Add operator primitive for mindspore.ops.AdaptiveMaxPool3D.
• [STABLE] Add operator primitive for mindspore.ops.AdjustHue.
• [STABLE] Add operator primitive for mindspore.ops.BartlettWindow.
• [STABLE] Add operator primitive for mindspore.ops.BesselJ0.
• [STABLE] Add operator primitive for mindspore.ops.BesselJ1.
• [STABLE] Add operator primitive for mindspore.ops.BesselK0.
• [STABLE] Add operator primitive for mindspore.ops.BesselK0e.
• [STABLE] Add operator primitive for mindspore.ops.BesselK1.
• [STABLE] Add operator primitive for mindspore.ops.BesselK1e.
• [STABLE] Add operator primitive for mindspore.ops.BesselY0.
• [STABLE] Add operator primitive for mindspore.ops.BesselY1.
• [STABLE] Add operator primitive for mindspore.ops.Betainc.
• [STABLE] Add operator primitive for mindspore.ops.Bincount.
• [STABLE] Add operator primitive for mindspore.ops.BlackmanWindow.
• [STABLE] Add operator primitive for mindspore.ops.Bucketize.
• [STABLE] Add operator primitive for mindspore.ops.CombinedNonMaxSuppression.
• [STABLE] Add operator primitive for mindspore.ops.CompareAndBitpack.
• [STABLE] Add operator primitive for mindspore.ops.Complex.
• [STABLE] Add operator primitive for mindspore.ops.DataFormatVecPermute.
• [STABLE] Add operator primitive for mindspore.ops.EuclideanNorm.
• [STABLE] Add operator primitive for mindspore.ops.Expand.
• [STABLE] Add operator primitive for mindspore.ops.ExtractGlimpse.
• [STABLE] Add operator primitive for mindspore.ops.FillDiagonal.
• [STABLE] Add operator primitive for mindspore.ops.FractionalAvgPool.
• [STABLE] Add operator primitive for mindspore.ops.FractionalMaxPool.
• [STABLE] Add operator primitive for mindspore.ops.Gcd.
• [STABLE] Add operator primitive for mindspore.ops.HammingWindow.
• [STABLE] Add operator primitive for mindspore.ops.Histogram.
• [STABLE] Add operator primitive for mindspore.ops.HSVToRGB.
• [STABLE] Add operator primitive for mindspore.ops.Lcm.
• [STABLE] Add operator primitive for mindspore.ops.LeftShift.
• [STABLE] Add operator primitive for mindspore.ops.ListDiff.
• [STABLE] Add operator primitive for mindspore.ops.LogSpace.
• [STABLE] Add operator primitive for mindspore.ops.Lstsq.
• [STABLE] Add operator primitive for mindspore.ops.MatrixDiagPartV3.
• [STABLE] Add operator primitive for mindspore.ops.MatrixDiagV3.
• [STABLE] Add operator primitive for mindspore.ops.MatrixExp.
• [STABLE] Add operator primitive for mindspore.ops.MatrixPower.
• [STABLE] Add operator primitive for mindspore.ops.MaxPool3DWithArgmax.
• [STABLE] Add operator primitive for mindspore.ops.MaxUnpool2D.
• [STABLE] Add operator primitive for mindspore.ops.MultilabelMarginLoss.
• [STABLE] Add operator primitive for mindspore.ops.NextAfter.
• [STABLE] Add operator primitive for mindspore.ops.Orgqr.
• [STABLE] Add operator primitive for mindspore.ops.ReduceStd.
• [STABLE] Add operator primitive for mindspore.ops.RGBToHSV.
• [STABLE] Add operator primitive for mindspore.ops.RightShift.
• [STABLE] Add operator primitive for mindspore.ops.SampleDistortedBoundingBoxV2.
• [STABLE] Add operator primitive for mindspore.ops.ScaleAndTranslate.
• [STABLE] Add operator primitive for mindspore.ops.ScatterAddWithAxis.
• [STABLE] Add operator primitive for mindspore.ops.ScatterNdDiv.
• [STABLE] Add operator primitive for mindspore.ops.ScatterNdMax.
• [STABLE] Add operator primitive for mindspore.ops.ScatterNdMul.
• [STABLE] Add operator primitive for mindspore.ops.STFT.
• [STABLE] Add operator primitive for mindspore.ops.Trace.
• [STABLE] Add operator primitive for mindspore.ops.UpsampleNearest3D.
• [STABLE] Add operator primitive for mindspore.ops.UpsampleTrilinear3D.
• [STABLE] Add distributed weight conversion interface mindspore.parallel.transform_checkpoints.
• [STABLE] Add distributed weight conversion interface mindspore.parallel.transform_checkpoint_by_rank.

Backwards Incompatible Change

Python API

• The mindspore.ms_function interface is renamed to mindspore.jit, and mindspore.ms_function will be deprecated and removed in a future version.
• The mindspore.ms_class interface is renamed to mindspore.jit_class, and mindspore.ms_class will be deprecated and removed in a future version.
• The mindspore.ops.ms_kernel interface is renamed to mindspore.ops.kernel, and mindspore.ops.ms_kernel will be deprecated and removed in a future version.
• The mindspore.dataset.map interface parameter column_order does not take effect, usemindspore.dataset.project.
• The mindspore.dataset.close_pool and mindspore.dataset.to_device and mindspore.dataset.set_dynamic_columns are deprecated and removed in this version.

Bug fixes

• Fixed an issue where the mixed precision functional interface could not modify the backend driver in graph mode
• Fixed the problem that users can automatically transfer device_id in the single-P scenario for the following networks:（mobilenetv1/fasterrcnn/yolov3/yolov4/yolov5/unet/openpose/simplepose/crnn/gnmtv2/faceattribute/facequality/facedetection）

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking, shu-kun-zhang.

Contributions of any kind are welcome!

MindSpore 1.10.1 Release Notes

Bug fixes

• Fixed the issue that the specified axis is not considered in logsumexp anti-overflow processing
• Fixed the compilation dependency of proto file
• Fixed the issue that the print operator printing result is not normal
• Fixed the issue that the equal operator is out of range
• Fixed the problem that when function wrapped by @jit，the cell id is not correct
• Fixed the GNN scenario data type verification error
• Fixed the problem that the dataset.map multi-process degenerates into threads

Contributors

Thanks goes to these wonderful people:

archer2049, caifubi, chenfei_mindspore, gaoshuanglong, Greatpan, guozhijian, huoxinyou, Kxiong, lanzhineng, lijunbin, liubuyu, liuchuting, luochao60, lyqlola, nomindcarry, TuDouNi, xiaotianci, xupan, yangshuo, yefeng, YingtongHu, yuchaojie, zhoufeng, ZPaC, 刘勇琪, 吕昱峰, 王禹程, 于振华.

Contributions of any kind are welcome!

MindSpore Release Notes

查看中文

MindSpore 1.10.0 Release Notes

Major Features and Improvements

DataSet

• [STABLE]The timeout waiting time is adjusted in data sinking mode. The default value is 600s after adjusted. This solves the isuses that the GetNext operator may timeout due to environment resource competition and large computing workload when training in sink mode.

Bug fixes

• Fixed an issue where some Primitive operators in AMP cannot be instantiated in graph mode and the interface is unavailable.
• Fixed an issue of DynamicRNN execution failure in LSTM network under the scenario of computational force segmentation on Ascend platform.
• Fixed DEVICE_ID cannot be set by single card train scripts parameters in mobilenet, fasterrcnn, yolo, etc.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking, shu-kun-zhang.

Contributions of any kind are welcome!

MindSpore Lite 1.10.0 Release Notes

Bug fixes

• Fixed potential accuracy problem of arithmetic type CPU kernels at dynamical shape case.
• Fixed the Incorrect Write Address of the Deconv Quantization Operator.

MindSpore 1.9.0 Release Notes

Major Features and Improvements

FrontEnd

• [STABLE] Add the object-oriented and functional combination programming paradigm, add mixed-precision APIs for combination programming paradigms such as mindspore.amp.LossScaler, mindspore.amp.DynamicLossScaler, mindspore.amp.StaticLossScaler, mindspore.amp.auto_mixed_precision and mindspore.amp.all_finite.

API Change

operator

• [STABLE] Add nn interface for nn.AdaptiveAvgPool3d.
• [STABLE] Add functional interface for ops.adaptive_avg_pool3d.
• [STABLE] Add functional interface for ops.addcdiv.
• [STABLE] Add functional interface for ops.addcmul.
• [STABLE] Add GPU and CPU support for ops.approximate_equal.
• [STABLE] Add GPU support for ops.atanh.
• [STABLE] Add GPU support for ops.bessel_i0.
• [STABLE] Add Ascend support for ops.bessel_i0e.
• [STABLE] Add GPU support for ops.bessel_i1.
• [STABLE] Add Ascend and GPU support for ops.bessel_i1e.
• [STABLE] Add GPU support for ops.bessel_j0.
• [STABLE] Add GPU support for ops.bessel_j1.
• [STABLE] Add GPU support for ops.bessel_k0.
• [STABLE] Add GPU support for ops.bessel_k0e.
• [STABLE] Add GPU support for ops.bessel_k1.
• [STABLE] Add GPU support for ops.bessel_k1e.
• [STABLE] Add GPU support for ops.bessel_y0.
• [STABLE] Add GPU support for ops.bessel_y1.
• [STABLE] Add functional interface for ops.bias_add.
• [STABLE] Add GPU support for ops.bitwise_and.
• [STABLE] Add GPU support for ops.bitwise_or.
• [STABLE] Add GPU support for ops.bitwise_xor.
• [STABLE] Add Ascend support for ops.grid_sample.
• [STABLE] Add CPU support for ops.inplace_update.
• [STABLE] Add Ascend and GPU support for ops.isclose.
• [STABLE] Add Ascend support for ops.isnan.
• [STABLE] Add GPU support for ops.lerp.
• [STABLE] Add functional interface for ops.random_poisson.
• [STABLE] Add functional interface for ops.reverse_sequence.
• [STABLE] Add GPU support for ops.scatter_mul.
• [STABLE] Add functional interface for ops.scatter_nd_max.
• [STABLE] Add functional interface for ops.scatter_nd_min.
• [STABLE] Add GPU support for ops.SparseToDense.
• [STABLE] Add functional interface for ops.square.
• [STABLE] Add GPU support for ops.standard_laplace.
• [STABLE] Add functional interface for ops.std.
• [STABLE] Add Ascend and GPU support for ops.trunc.
• [STABLE] Add functional interface for ops.unsorted_segment_sum.
• [STABLE] Add functional interface for ops.xdivy.
• [STABLE] Add GPU support for ops.xlogy.
• Deprecate ops.poisson and use ops.random_poisson instead.
• Deprecate ops.SparseApplyAdagrad and use ops.SparseApplyAdagradV2 instead.

Bug fixes

• [BUGFIX] The logic of the auto mixed precision (amp) O2 level is revised. In addition to the BatchNorm1d and BatchNorm2d operators, the other two operators BatchNorm3d and LayerNorm are added. The four operators still use the float32 data type when calculating.

• [BUGFIX] Fix the problem that when processing string type data, if output_numpy=True is specified when calling the create_dict_iterator or create_tuple_iterator interface, the obtained data will be of type numpy.bytes_. After this fixing, these interfaces will directly return numpy.str_ type data, and users do not need to perform string decoding operations on it. Likewise, when performing user defined processing functions, the received data will also be of type numpy.str_ directly, matching the original source data type.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, liyanliu, lizhenyu, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, panfengfeng, panyifeng, Payne, peixu_ren, Pengyongrong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanyuan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking, shu-kun-zhang.

Contributions of any kind are welcome!

MindSpore 1.7.1 Release Notes

New Features

• [STABLE] Support custom back propagation function for cell which uses parameters in the forward part.
• [BETA] Support the SNN structure, and add demo on lenet and resnet50.
• [STABLE] Support predicting the binding affinity network Pafnucy for protein-ligand complexes on Ascend.

Bug fixes

• Fix the bug that the AICPU operator could not be found in some scenarios when the profiler parsed the performance data.
• Modify the function of dying gasp, and save ckpt file name error bug.
• Fix the derivative problem when the tuple input of an operator contains parameter. For example, the concat operator.
• Fix the shard of expert bias in pipline with moe scenario.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking.

Contributions of any kind are welcome!

MindSpore 1.8.0

MindSpore 1.8.0 Release Notes

Major Features and Improvements

FrontEnd

• [BETA] Add mindspore.Model.fit API, add mindspore.callback.EarlyStopping and mindspore.callback.ReduceLROnPlateau in Callback.
• [BETA] Support custom operator implemented by Julia.
• [BETA] Support custom operator implemented by MindSpore Hybrid DSL.
• [STABLE] The export() interface supports the export of a model using a custom encryption algorithm, and the load() interface supports the import of a model using a custom decryption algorithm.
• [BETA] [Unified_Dynamic_and_Static_Graphs] [Usability] Constant-type data (tuple/list/dict is supported in Version 1.8) can be set to be variable during graph compiling.
• [BETA] [Unified_Dynamic_and_Static_Graphs] JIT fallback is used to support the control flow capability in the constant scenario.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The Python raise statement is supported in the graph mode constant scenario.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The Python assert statement is supported in the graph mode constant scenario.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The Python print statement is supported in the graph mode constant scenario.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The str.format() method is supported in the graph mode.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The slice method can be used to assign a value to the list in the graph mode.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] The instances of custom classes can be created and invoked in the graph mode.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] Obtaining the properties of a class from the Cell array and the custom class array is supported.
• [STABLE] [Unified_Dynamic_and_Static_Graphs] isinstance supports scenario expanding in the graph mode.
• [STABLE] Rename the custom operator decorator 'ms_hybrid' to 'ms_kernel'.
• [BETA] Custom operator Hybrid DSL is supported on the backend of CPU.
• [BETA] Custom operator Ascend backend adds custom scheduling primitive syntax support.

PyNative

• [STABLE] Implement the AdamWeightDecay operator to replace the original small operator combination mode.
• [STABLE] In PyNative mode, execute the optimizer by unifying the dynamic and static graphs.
• [STABLE] Optimize the execution performance of PyNative bprop graph and ms_function.

Auto Parallel

• [STABLE] Docking the AllToAll single-operator mode. Support AllToAll Operator in the KernelByKernel execution mode.
• [STABLE] Whole-graph offloading supports MPI launching. In Whole-graph offloading, launching with MPI is supported.
• [STABLE] Seeds of model weights provide parallel interface configuration. If you do not set the random number of seeds through the mindspore.set_seed command, the weights initialized by each parameter is determined by the current fragment index. If the random number of seeds are configured, the initialization results of the same shape and weight of the same segmentation policy are the same.
• [STABLE] The HCCL shields internal full-mesh and non-full-mesh connections. Both fully-connected AllToAllv and hierarchical AllToAllv are allowed in one training session.
• [BETA] CPU optimizer fusion. Multiple optimizer operators are combined according to data types through cross-parameter fusion, improving performance. Currently, It has been verified on CPU AdamWeightDecay optimizer. You can use the flatten_weights method in the network cell class to enable this function.

Executor

• [STABLE] Provide southbound API.
• [STABLE] Multi-actor fusion execution to optimize the execution performance during runtime.
• [STABLE] Nopop operators (eg. reshape) execute elimination.
• [STABLE] Embedded cache architecture switches unified distributed runtime.
• [STABLE] Parameter Server switches unified distributed runtime.
• [STABLE] Support Parameter Server mode training on CPU.

DataSet

• [STABLE] When using the map operation for dataset objects and the parameters like: num_parallel_workers > 1 and python_multiprocessing=True, the multi-process mechanism is optimized, so that the data channel and child processes are mapped one by one, avoiding excessive file handle occupation, and closing_pool interface is also deleted.
• [STABLE] Add a batch of Vision, Text and Audio data augmentation operations.
• [STABLE] Fix a bug where the flat_map method of the Dataset class does not flatten the result.
• [STABLE] Unify import paths of dataset augmentation APIs to provide more easier way to use. Refer to latest api usages.

API Change

Backwards Incompatible Change

Python API

• DVPP simulation algorithm is no longer supported. Remove mindspore.dataset.vision.c_transforms.SoftDvppDecodeRandomCropResizeJpeg and mindspore.dataset.vision.c_transforms.SoftDvppDecodeResizeJpeg interfaces.
• Add on_train_epoch_end method in LossMonitor, which implements printing metric information in the epoch level when it is used in mindspore.Model.fit.
• TimeMonitor printing content changes, and the printed content is added to "train" or "eval" to distinguish between training and inference phases.
• filter_prefix of mindspore.load_checkpoint interface: empty string ("") is no longer supported, and the matching rules are changed from strong matching to fuzzy matching.

MindSpore Lite

Major Features and Improvements

API

• [STABLE] Add C++ and Python APIs for model conversion.
• [STABLE] Add Python APIs for model inference.

Post-Training Quantization

• [STABLE] Support perlayer quantization, and built-in CLE to optimize perlayer quantization accuracy.

operator

• [STABLE] Add GPU support for ops.adaptive_avg_pool2d.
• [BETA] Add Ascend, GPU, and CPU support for ops.adaptive_max_pool2d .
• [BETA] Add CPU support for ops.approximate_equal.
• [STABLE] Add CPU support for ops.argmin.
• [BETA] Add CPU support for ops.assign_sub.
• [STABLE] Add GPU support for ops.bernoulli.
• [BETA] Add CPU support for ops.bessel_i0.
• [BETA] Add CPU support for ops.bessel_i0e.
• [BETA] Add CPU support for ops.bessel_i1.
• [BETA] Add CPU support for ops.bessel_i1e Add CPU support.
• [STABLE] Add CPU support for ops.bessel_j0.
• [STABLE] Add CPU support for ops.bessel_j1.
• [STABLE] Add CPU support for ops.bessel_k0.
• [STABLE] Add CPU support for ops.bessel_k0e.
• [BETA] Add CPU support for ops.bessel_k1.
• [BETA] Add CPU support for ops.bessel_k1e.
• [STABLE] Add CPU support for ops.bessel_y0.
• [STABLE] Add CPU support for ops.bessel_y1.
• [STABLE] Add CPU support for ops.bitwise_and.
• [STABLE] Add CPU support for ops.bitwise_or.
• [STABLE] Add CPU support for ops.bitwise_xor.
• [STABLE] Add functional interface for ops.broadcast_to.
• [BETA] Add GPU and CPU support for ops.ceil.
• [BETA] Add GPU support for ops.col2im.
• [BETA] Add functional interface for ops.concat.
• [STABLE] Add GPU support for ops.cosh.
• [STABLE] Add Ascend and CPU support for ops.ctc_greedy_decoder.
• [BETA] Add GPU and CPU support for ops.DataFormatDimMap.
• [BETA] Add GPU and CPU support for ops.dropout2d.
• [BETA] Add CPU support for ops.dropout3d.
• [BETA] Add CPU support for ops.erf.
• [BETA] Add CPU support for ops.erfc.
• [STABLE] Add functional interface for ops.expand_dims.
• [STABLE] Add GPU and CPU support for ops.fast_gelu.
• [STABLE] Add Ascend dynamic shape support for ops.flatten.
• [BETA] Add GPU and CPU support for ops.ger.
• [STABLE] Add Ascend, GPU, and CPU support for ops.gumbel_softmax.
• [BETA] Add GPU and CPU support for ops.hardshrink.
• [BETA] Add CPU support for ops.index_add.
• [BETA] Add CPU support for ops.inplace_add.
• [BETA] Add CPU support for ops.inplace_sub.
• [STABLE] Add CPU support for ops.intopk.
• [STABLE] Add GPU and CPU support for ops.inv.
• [STABLE] Add GPU and CPU support for ops.invert.
• [BETA] Add CPU support for ops.isclose.
• [STABLE] Add CPU support for ops.lerp.
• [BETA] Add CPU support for ops.linspace.
• [BETA] Add functional interface for ops.log_softmax.
• [BETA] Add Ascend, GPU, and CPU support for ops.norm.
• [BETA] Add CPU support for ops.lrn.
• [BETA] Add GPU support for ops.masked_select.
• [BETA] Add GPU and CPU support for ops.matrix_band_part.
• [BETA] Add GPU and CPU support for ops.matrix_solve.
• [BETA] Add CPU support for ops.meshgrid.
• [STABLE] Add CPU support for ops.mish.
• [BETA] Add GPU support forops.nonzero.
• [STABLE] Add GPU and CPU support for ops.padding.
• [BETA] Add Ascend dynamic shape support for ops.pow.
• [BETA] Add functional interface for ops.range.
• [BETA] Add Ascend dynamic shape support for ops.round.
• [STABLE] Add Ascend dynamic shape support for ops.scatter_add.
• [STABLE] Add Ascend dynamic shape support for ops.scatter_div.
• [BETA] Add GPU support for ops.scatter_max.
• [BETA] Add GPU support for ops.scatter_min.
• [BETA] Add CPU support for ops.scatter_nd_add.
• [STABLE] Add GPU and CPU support for ops.scatter_nd_div.
• [STABLE] Add GPU and CPU support for ops.scatter_nd_min.
• [STABLE] Add GPU and CPU support for ops.scatter_nd_mul.
• [BETA] Add CPU support for ops.scatter_nd_sub.
• [STABLE] Add Ascend dynamic shape support for ops.scatter_update.
• [BETA] Add Ascend dynamic shape support for ops.select.
• [BETA] Add GPU and CPU support for ops.selu.
• [BETA] Add GPU and CPU support for ops.soft_shrink.
• [BETA] Add CPU support for ops.softsign.
• [STABLE] Add GPU support for ops.tan.
• [BETA] Add Ascend and CPU support ops.tensor_scatter_add.
• [STABLE] Add GPU and CPU support for ops.tensor_scatter_div.
• [STABLE] Add GPU and CPU support for ops.tensor_scatter_mul.
• [BETA] Add Ascend and CPU support for ops.tensor_scatter_sub.
• [STABLE] Add Ascend, GPU, and CPU support for nn.AdaptiveAvgPool1d.
• [STABLE] Add Ascend, GPU, and CPU support for nn.AdaptiveMaxPool1d.
• [BETA] Add Ascend, GPU, and CPU support for nn.BiDense.
• [STABLE] Add Ascend, GPU, and CPU support for nn.ConstantPad1d.
• [STABLE] Add Ascend, GPU, and CPU support for nn.ConstantPad2d.
• [STABLE] Add Ascend, GPU, and CPU support for nn.ConstantPad3d.
• [STABLE] Add Ascend, GPU, and CPU support for nn.Hardtanh.
• [STABLE] Add Ascend, GPU, and CPU support for nn.HuberLoss.
• [STABLE] Add Ascend, GPU, and CPU support for nn.RReLU.
• [STABLE] Add Ascend, GPU, and CPU support for nn.Tanhshrink.
• [STABLE] Add Ascend, GPU, and CPU support for nn.Threshold.
• [STABLE] Add Ascend, GPU, and CPU support for nn.ZeroPad2d.
• [BETA] Add GPU support for ops.unique_consecutive.
• [STABLE] Add CPU support for ops.unsorted_segment_max.
• [STABLE] Add CPU support for ops.unsorted_segment_min.
• [STABLE] Add GPU support for ops.unsorted_segment_prod.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking, shu-kun-zhang.

Contributions of any kind are welcome!

MindSpore 1.7.0

MindSpore 1.7.0 Release Notes

Major Features and Improvements

OS

• [STABLE] Support Python 3.8 (Linux/Windows/Mac).
• [STABLE] Installation improved with more detailed install guide and automated shell scripts.
• [STABLE] Support operator computing with multi-thread under Windows.
• [STABLE] Compatible with GCC from version 7.3 to 9.x.

FrontEnd

• [STABLE] Support dynamic weight decay for optimizers, that is weight decay value will change according to the increasing step during training.
• [STABLE] Add four methods to create Tensor, which are mindspore.numpy.rand(), mindspore.numpy.randn(), mindspore.numpy.randint(), and mindspore.ops.arange().
• [STABLE] Add mindspore.train.callback.History in Callback.
• [BETA] Support custom operator implemented by Julia operator.
• [STABLE] Support accessing attributes and methods of user-defined classes through mindspore.ms_class class decorator.
• [STABLE] Support training when a network has side effect operations and control flow statements at the same time.
• [STABLE] Support for more complex control flow syntax, such as a for loop statement in the body of a while loop.
• [STABLE] Improve the performance of networks with complex syntax control flow statements by decreasing the num of subgraphs.

PyNative

• [STABLE] Add Hook functions in PyNative mode, including register_forward_pre_hook, register_forward_hook of the forward hook interface, register_backward_hook of the reverse hook interface.
• [STABLE] Optimize the execution performance of PyNative mode, and execute the front-end Python and the back-end C++ in parallel.

Auto Parallel

• [STABLE] Support TopK routing, data parallel and optimizer state parallel when enable MoE.
• [STABLE] Support AllGather/ReduceScatter communication operator fusion. Support AllReuduce fusion by the data volume size in DATA_PARALLEL mode.
• [STABLE] Support ops.clip_by_global_norm in the parallel mode.
• [STABLE] Support AdaSum optimizer in the parallel mode.
• [STABLE] Support automatic optimizer state parallel.
• [STABLE] Support AlltoAll configurable. Support automatically add virtualdataset cell.
• [STABLE] Support automatically inference trainable parameters in pipeline parallel training.
• [STABLE] Support clusters where the device number is not the power of 2.
• [STABLE] Support sharding propagation in auto-parallel mode.
• [STABLE] Support optimizer offload under the unified runtime.
• [STABLE] Support Adafactor operator on CPU.
• [STABLE] Support sharding at H/W axis for Conv2d/Conv2DTranspose operator. Support operators such as ResizeBilinear，ROIAlign, CropAndResize, BoundingBoxEncode, IOU and RandomChoiceWithMask.

Executor

• [BETA] Failure Recovery Under Data Parallel Training Support auto failure recovery under data parallel training mode.
• [BETA] Support searching for the number of threads under the CPU to obtain the optimal number of threads for execution. The entire search process takes 50 steps, and the overall performance will reach a stable state after 50 steps. When testing performance, data after 50 steps need to be used as a standard.

DataSet

• [STABLE] Add dataset operations mapping between TensorFlow.data module and MindSpore.dataset module, check list.
• [STABLE] Python multiprocessing optimization and make processes exit normally.
• [STABLE] Support Dataset Autotune for tuning the speed of dataset pipeline automatically.
• [BETA] Dataset Offload support new data augmentation operations: RandomColorAdjust, RandomSharpness, TypeCast.
• Output a single data column when __getitem__/__next__ methods of GeneratorDataset return a single NumPy object.
• Use ulimit -u 10240 to increase the number of threads/processes available to the current user when specify too many processes or threads for loading dataset may cause RuntimeError: can't start new thread.

API Change

Backwards Incompatible Change

Python API

• Modify the gradient return value type of the hook corresponding to the register_backward_hook function, and change the gradient return value to the tuple type uniformly.(!31876:Change single ret to tuple ret for pynative cell hook)
• Deprecated usage: import mindspore.dataset.engine.datasets as ds. Use import mindspore.dataset as ds instead as recommended in mindspore doc.
• Add mindspore.ms_class interface, as class decorator for user-defined classes. It allows MindSpore to identify user-defined classes and access their attributes and methods(!30855:Support user-defined classes by ms_class decorators)
• Deprecate mindspore.SparseTensor and use mindspore.COOTensor instead. (!28505:Change SparseTensor to COOTensor)

MindSpore Lite

Major Features and Improvements

Post quantization

• [STABLE] Support post quantization to run dynamic quantization algorithm.
• [BETA] Support post quantized model to run on NVIDIA GPU.

Contributors

Thanks goes to these wonderful people:

AGroupofProbiotocs, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hesham, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Jiabin Liu, jianghui58, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, liuyongqi, laiyongqiang, leonwanghui, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, lvchangquan, lvliang, lz, maning202007, Margaret_wangrui, mengyuanli, Ming_blue, ms_yan, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking.

Contributions of any kind are welcome!

MindSpore 1.6.1

MindSpore 1.6.1 Release Notes

Major Features and Improvements

Executor

• [STABLE] Side effect operators are supported in the control flow training scenario.

Bug fixes

• Fixed the problem that the accuracy of the transformer is reduced.

Contributors

Thanks goes to these wonderful people:

Adel, AGroupofProbiotocs, anthonyaje, anzhengqi, askmiao, baihuawei, baiyangfan, bai-yangfan, bingyaweng, BowenK, buxue, caifubi, CaoJian, caojian05, caozhou, Cathy, changzherui, chenbo116, chenfei, chengxianbin, chenhaozhe, chenjianping, chenzomi, chenzupeng, chujinjin, cj, cjh9368, Corleone, damon0626, danish, Danish, davidmc, dayschan, doitH, dong-li001, eric, Eric, fary86, fuzhiye, Gaoxiong, GAO_HYP_XYJ, gengdongjie, Gogery, gongdaguo, gray0v0, gukecai, guoqi, gzhcv, hangq, hanhuifeng2020, Harshvardhan, He, heleiwang, hexia, Hoai, HuangBingjian, huangdongrun, huanghui, huangxinjing, huqi, huzhifeng, hwjiaorui, Islam Amin, Jesse, , Jiabin Liu, jianghui58, jiangzhiwen, Jiaqi, jin-xiulang, jinyaohui, jjfeing, John, Jonathan, jonyguo, JulyAi, jzg, kai00, kingfo, kingxian, kpy, kswang, laiyongqiang, leonwanghui, Li, liangchenghui, liangzelang, lichen_101010, lichenever, lihongkang, lilei, limingqi107, ling, linqingke, Lin Xh, liubuyu, liuwenhao4, liuxiao78, liuxiao93, liuyang_655, liuzhongkai, Lixia, lixian, liyanliu, liyong, lizhenyu, luopengting, luoyang, lvchangquan, lvliang, lz, mahdi, Mahdi, maning202007, Margaret_wangrui, mayang, mengyuanli, Ming_blue, nhussain, ougongchang, panfengfeng, panyifeng, Payne, Peilin, peixu_ren, Pengyongrong, qianlong, qianjiahong, r1chardf1d0, riemann_penn, rmdyh, Sheng, shenwei41, simson, Simson, Su, sunsuodong, tao_yunhao, tinazhang, VectorSL, , Wan, wandongdong, wangdongxu, wangmin, wangnan39@huawei.com, wangyue01, wangzhe, wanyiming, Wei, wenchunjiang, wilfChen, WilliamLian, wsc, wudenggang, wukesong, wuweikang, wuxuejian, Xiao Tianci, Xiaoda, xiefangqi, xinyunfan, xuanyue, xulei2020, Xun, xuyongfei, yanghaitao, yanghaitao1, yanghaoran, YangLuo, yangruoqi713, yankai, yanzhenxiang2020, yao_yf, yepei6, yeyunpeng, Yi, yoni, yoonlee666, yuchaojie, yujianfeng, yuximiao, zengzitao, Zhang, zhanghaibo5@huawei.com, zhanghuiyao, zhanghui_china, zhangxinfeng3, zhangyihui, zhangz0911gm, zhanke, zhanyuan, zhaodezan, zhaojichen, zhaoting, zhaozhenlong, zhengjun10, Zhenglong Li, zhiqwang, zhoufeng, zhousiyi, zhouyaqiang, zhouyifengCode, Zichun, Zirui, Ziyan, zjun, ZPaC, wangfengwfwf, zymaa, gerayking.

Contributions of any kind are welcome!