golden-stick 标签

v0.6.0

### Major Features and Improvements

* The `RoundToNearest` supports Mindformers' KVCache int8 quantization now, i.e. `PagedAttentionMgr` class, mainly for Llama2 networks.
* Added Post-Training Quantization algorithm named `PTQ` which supports SmoothQuant, A16W8, KVCacheInt8 and their combinations, such as A16W8 combined with KVCacheInt8, SmoothQuant combined with KVCacheInt8, etc., and the corresponding algorithm capabilities can be obtained by configuring PTQConfig. The algorithm is mainly supports ParallelLlama2 network from the MindFormers community.

### API Change

* `PTQConfig` adds the following three parameters:
    * `act_quant_dtype`: The data type is mindspore.dtype. The default value is None. The options and meanings are as follows:

|  act_quant_dtype  | mindspore.dtype.int8  | None(default)  |
        |  ----  | ----  | ----  |
        | meanings  | quantize input to int8 | does not quantize input |
    * `weight_quant_dtype`: The data type is mindspore.dtype. The default value is mindspore.dtype.int8. The options and meanings are as follows:

|  weight_quant_dtype  | mindspore.dtype.int8(default)  | None  |
        |  ----  | ----  | ----  |
        | meanings  | quantize weights to int8 | does not quantize weights |
    * `kvcache_quant_dtype`: The data type is mindspore.dtype. The default value is None. The options and meanings are as follows:

|  kvcache_quant_dtype  | mindspore.dtype.int8  | None（default）  |
        |  ----  | ----  | ----  |
        | meanings  | quantize kvcache to int8 | does not quantize kvcache |
    * `outliers_suppression`: The data type is OutliersSuppressionType. The default value is OutliersSuppressionType.NONE. The options and meanings are as follows:

|  outliers_suppression  | OutliersSuppressionType.SMOOTH  | OutliersSuppressionType.NONE（default）  |
        |  ----  | ----  | ----  |
        | meanings  | employ smooth approach to suppress outliers in activation and weight | does not suppress outliers |

### Contributors

Thanks goes to these wonderful people:

ccsszz, yyyyrf, hangangqiang

Contributions of any kind are welcome!

92b20ce

2024-10-16 12:08

下载查看发行版

v0.5.0

## MindSpore Golden Stick 0.5.0 Release Notes

### Major Features and Improvements

* [DEMO] Added post-training quantization W8A8 algorithm `SmoothQuant` mainly for Llama2 network.

### API Change

* Added `kwargs` to `apply` api of `CompAlgo` class as extensible parameter for subclasses.
* Added `NetworkHelper` abstract class as adapter for decoupling between algorithm and framework.
* Added `MFLlama2Helper` class as adapter between algorithm and MindFormers.
* [DEMO] Added `SmoothQuant` class as entry of SmoothQuant algorithm.
* Added parameter checking that `RoundToNearest` algorithm only supports BackendTarget.ASCEND as backend.

### Contributors

Thanks goes to these wonderful people:

ccsszz, yyyyrf, hangangqiang

Contributions of any kind are welcome!

## MindSpore Golden Stick 0.4.1 Release Notes

### Major Features and Improvements

* Optimize the time taken by RoundToNearest algorithm to quantify weights.
* Optimize the compilation time of RoundToNearest quantization network.

### Contributors

Thanks goes to these wonderful people:

ccsszz, yyyyrf, hangangqiang

Contributions of any kind are welcome!

c1cd03b