MMLSpark 标签 - Gitee.com

开源
企业版
高校版
搜索
帮助中心
使用条款
关于我们

开源企业版高校版私有云 Gitee AI ^NEW

代码拉取完成，页面将自动刷新

捐赠

捐赠前请先登录

取消前往登录

扫描微信二维码支付

取消

支付完成

支付提示

将跳转至支付宝完成支付

确定

取消

Watch

不关注关注所有动态仅关注版本发行动态关注但不提醒动态

1 Star 1 Fork 9

hgkdt/MMLSpark

forked from Gitee 极速下载/MMLSpark

确定同步？

同步操作将从 Gitee 极速下载/MMLSpark 强制同步，此操作会覆盖自 Fork 仓库以来所做的任何修改，且无法恢复！！！

确定后同步将在后台操作，完成时将刷新页面，请耐心等待。

删除在远程仓库中不存在的分支和标签

同步 Wiki （当前仓库的 wiki 将会被覆盖！）

取消

确定

代码 Wiki 统计流水线

服务

Jenkins for gitee

Jenkins for Gitee

腾讯云托管

Cloud serverless

腾讯云 Serverless

标签名

描述

提交信息

操作

- Add the `TagImage` and `DescribeImage` services
- Add Ranking Cross Validator and Evaluator

- Learn hopw to use HTTP on Spark to work with arbitrary web apis in [HttpOnSpark - Working with Arbitrary Web APIs.ipynb](https://github.com/Azure/mmlspark/blob/master/notebooks/samples/HttpOnSpark%20-%20Working%20with%20Arbitrary%20Web%20APIs.ipynb)

- Fix issue with `raw2probabilityInPlace`
- Add weight column
- Add `getModel` API to `TrainClassifier` and `TrainRegressor`
- Improve robustness of getting executor cores

- Improve robustness of Gateway creation and management
- Imrpove Gateway documentation

- Updated to Spark 2.4.0
- LightGBM version update to 2.1.250

- Fix Flaky Tests
- Remove autogeneration of scalastyle
- Increase training dataset size in snow leopard example

We would like to acknowledge the developers and contributors, both internal and external who helped create this version of MMLSpark.

- Ilya Matiach, Casey Hong, Karthik Rajendran, Daniel Ciborowski, Sebastien Thomas, Eli Barzilay,  Sudarshan Raghunathan, @flybywind, @wentongxin, @haal

2018-11-30 04:26

- Add the `TagImage` and `DescribeImage` services
- Add Ranking Cross Validator and Evaluator

- Learn hopw to use HTTP on Spark to work with arbitrary web apis in [HttpOnSpark - Working with Arbitrary Web APIs.ipynb](https://github.com/Azure/mmlspark/blob/master/notebooks/samples/HttpOnSpark%20-%20Working%20with%20Arbitrary%20Web%20APIs.ipynb)

- Fix issue with `raw2probabilityInPlace`
- Add weight column
- Add `getModel` API to `TrainClassifier` and `TrainRegressor`
- Improve robustness of getting executor cores

- Improve robustness of Gateway creation and management
- Imrpove Gateway documentation

- Updated to Spark 2.4.0
- LightGBM version update to 2.1.250

- Fix Flaky Tests
- Remove autogeneration of scalastyle
- Increase training dataset size in snow leopard example

We would like to acknowledge the developers and contributors, both internal and external who helped create this version of MMLSpark.

- Ilya Matiach, Casey Hong, Karthik Rajendran, Daniel Ciborowski, Sebastien Thomas, Eli Barzilay,  Sudarshan Raghunathan, @flybywind, @wentongxin, @haal

2018-11-30 04:26

- The Cognitive Services on Spark: A simple and scalable integration between the Microsoft Cognitive Services and SparkML
	- Bing Image Search
	- Computer Vision: OCR, Recognize Text, Recognize Domain Specific Content,
		 Analyze Image, Generate Thumbnails
	- Text Analytics: Language Detector, Entity Detector, Key Phrase Extractor,
		Sentiment Detector, Named Entity Recognition
    - Face: Detect, Find Similar, Identify, Group, Verify
- Added distributed model interpretability with LIME on Spark
- **100x** lower latencies (\<1ms) with Spark Serving
- Expanded Spark Serving to cover the full HTTP protocol
- Added the `SuperpixelTransformer` for segmenting images
- Added a Fluent API, `mlTransform` and `mlFit`,  for composing pipelines more elegantly

- Chain together cognitive services to understand the feelings of your favorite celebrities with `CognitiveServices - Celebrity Quote Analysis.ipynb`
- Explore how you can use Bing Image Search and Distributed Model Interpretability to get an Object Detection system without labeling any data in `ModelInterpretation - Snow Leopard Detection.ipynb`
- See how to deploy *any* spark computation as a Web service on *any* Spark platform with the `SparkServing - Deploying a Classifier.ipynb` notebook

- More APIs for loading LightGBM Native Models
- LightGBM training checkpointing and continuation
- Added tweedie variance power to LightGBM
- Added early stopping to lightGBM
- Added feature importances to LightGBM
- Added a PMML exporter for LightGBM on Spark

- Added the `VectorizableParam` for creating column parameterizable inputs
- Added `handler` parameter added to HTTP services
- HTTP on Spark now propagates nulls robustly

- Updated to Spark 2.3.1
- LightGBM version update to 2.1.250

- Added Vagrantfile for easy windows developer setup
- Improved Image Reader fault tolerance
- Reorganized Examples into Topics
- Generalized Image Featurizer and other Image based code to handle Binary Files as well as Spark Images
- Added `ModelDownloader` R wrapper
- Added `getBestModel` and `getBestModelInfo` to `TuneHyperparameters`
- Expanded Binary File Reading APIs
- Added `Explode` and `Lambda` transformers
- Added `SparkBindings` trait for automating spark binding creation
- Added retries and timeouts to `ModelDownloader`
- Added `ResizeImageTransformer` to remove `ImageFeaturizer` dependence on OpenCV

We would like to acknowledge the developers and contributors, both internal and external who helped create this version of MMLSpark. (In alphabetical order)

- Abhiram Eswaran, Anand Raman, Ari Green, Arvind Krishnaa Jagannathan, Ben Brodsky, Casey Hong, Courtney Cochrane, Henrik Frystyk Nielsen, Ilya Matiach, Janhavi Suresh Mahajan, Jaya Susan Mathew, Karthik Rajendran, Mario Inchiosa, Minsoo Thigpen, Soundar Srinivasan, Sudarshan Raghunathan,  @terrytangyuan

2018-09-23 13:50

- The Cognitive Services on Spark: A simple and scalable integration between the Microsoft Cognitive Services and SparkML
	- Bing Image Search
	- Computer Vision: OCR, Recognize Text, Recognize Domain Specific Content,
		 Analyze Image, Generate Thumbnails
	- Text Analytics: Language Detector, Entity Detector, Key Phrase Extractor,
		Sentiment Detector, Named Entity Recognition
    - Face: Detect, Find Similar, Identify, Group, Verify
- Added distributed model interpretability with LIME on Spark
- **100x** lower latencies (\<1ms) with Spark Serving
- Expanded Spark Serving to cover the full HTTP protocol
- Added the `SuperpixelTransformer` for segmenting images
- Added a Fluent API, `mlTransform` and `mlFit`,  for composing pipelines more elegantly

- Chain together cognitive services to understand the feelings of your favorite celebrities with `CognitiveServices - Celebrity Quote Analysis.ipynb`
- Explore how you can use Bing Image Search and Distributed Model Interpretability to get an Object Detection system without labeling any data in `ModelInterpretation - Snow Leopard Detection.ipynb`
- See how to deploy *any* spark computation as a Web service on *any* Spark platform with the `SparkServing - Deploying a Classifier.ipynb` notebook

- More APIs for loading LightGBM Native Models
- LightGBM training checkpointing and continuation
- Added tweedie variance power to LightGBM
- Added early stopping to lightGBM
- Added feature importances to LightGBM
- Added a PMML exporter for LightGBM on Spark

- Added the `VectorizableParam` for creating column parameterizable inputs
- Added `handler` parameter added to HTTP services
- HTTP on Spark now propagates nulls robustly

- Updated to Spark 2.3.1
- LightGBM version update to 2.1.250

- Added Vagrantfile for easy windows developer setup
- Improved Image Reader fault tolerance
- Reorganized Examples into Topics
- Generalized Image Featurizer and other Image based code to handle Binary Files as well as Spark Images
- Added `ModelDownloader` R wrapper
- Added `getBestModel` and `getBestModelInfo` to `TuneHyperparameters`
- Expanded Binary File Reading APIs
- Added `Explode` and `Lambda` transformers
- Added `SparkBindings` trait for automating spark binding creation
- Added retries and timeouts to `ModelDownloader`
- Added `ResizeImageTransformer` to remove `ImageFeaturizer` dependence on OpenCV

We would like to acknowledge the developers and contributors, both internal and external who helped create this version of MMLSpark. (In alphabetical order)

- Abhiram Eswaran, Anand Raman, Ari Green, Arvind Krishnaa Jagannathan, Ben Brodsky, Casey Hong, Courtney Cochrane, Henrik Frystyk Nielsen, Ilya Matiach, Janhavi Suresh Mahajan, Jaya Susan Mathew, Karthik Rajendran, Mario Inchiosa, Minsoo Thigpen, Soundar Srinivasan, Sudarshan Raghunathan,  @terrytangyuan

2018-09-23 13:50

v0.13

New Functionality:

* Export trained LightGBM models for evaluation outside of Spark.

* LightGBM on Spark supports multiple cores per executor

* `CNTKModel` works with multi-input multi-output models of any CNTK datatype

* Added Minibatching and Flattening transformers for adding flexible batching logic to pipelines, deep networks, and web clients.

* Added `Benchmark` test API for tracking model performance across versions

* Added `PartitionConsolidator` function for aggregating streaming data onto one partition per executor (for use with connection/rate-limited HTTP services)

Updates and Improvements:

* Updated to Spark 2.3.0

* Added Databricks notebook tests to build system

* `CNTKModel` uses significantly less memory

* Simplified example notebooks

* Simplified APIs for MMLSpark Serving

* Simplified APIs for CNTK on Spark

* LightGBM stability improvements

* `ComputeModelStatistics` stability improvements

Acknowledgements:

We would like to acknowledge the external contributors who helped create this version of MMLSpark (in order of commit history)

* 严伟,  @terrytangyuan, @ywskycn, @dvanasseldonk, Jilong Liao, @chappers, @ekaterina-sereda-rf

2018-06-29 02:25

v0.13

New Functionality:

* Export trained LightGBM models for evaluation outside of Spark.

* LightGBM on Spark supports multiple cores per executor

* `CNTKModel` works with multi-input multi-output models of any CNTK datatype

* Added Minibatching and Flattening transformers for adding flexible batching logic to pipelines, deep networks, and web clients.

* Added `Benchmark` test API for tracking model performance across versions

* Added `PartitionConsolidator` function for aggregating streaming data onto one partition per executor (for use with connection/rate-limited HTTP services)

Updates and Improvements:

* Updated to Spark 2.3.0

* Added Databricks notebook tests to build system

* `CNTKModel` uses significantly less memory

* Simplified example notebooks

* Simplified APIs for MMLSpark Serving

* Simplified APIs for CNTK on Spark

* LightGBM stability improvements

* `ComputeModelStatistics` stability improvements

Acknowledgements:

We would like to acknowledge the external contributors who helped create this version of MMLSpark (in order of commit history)

* 严伟,  @terrytangyuan, @ywskycn, @dvanasseldonk, Jilong Liao, @chappers, @ekaterina-sereda-rf

2018-06-29 02:25

v0.12

New functionality:

* MMLSpark Serving: a RESTful computation engine built on Spark
  streaming.  See `docs/mmlspark-serving.md` for details.

* New LightGBM Binary Classification and Regression learners and
  infrastructure with a Python notebook for examples.

* MMLSpark Clients: a general-purpose, distributed, and fault tolerant
  HTTP Library usable from Spark, Pyspark, and SparklyR.  See
  `docs/http.md`.

* Add `MinibatchTransformer` and `FlattenBatch` to enable efficient,
  buffered, minibatch processing in Spark.

* Added Python wrappers and a notebook example for the
  `TuneHyperparameters` module, demonstrating parallel distributed
  hyperparameter tuning through randomized grid search.

* Add a `MultiNGram` transformer for efficiently computing variable
  length n-grams.

* Added DataType parameter for building models that are parameterized by
  Spark data types.

Updates:

* Update per-instance statistics module so it works for any Spark ML
  estimators.

* Update CNTK to version 2.4.

* Updated Spark to version v2.2.1 (the following release is likely to be
  based on Spark 2.3).

* Also updated SBT and JVM.

* Refactored readers directory into `io` directory

Improvements:

* Fix the Conda installation in our Docker image, resolving issues with
  importing `numpy`.

* Fix a regression in R wrappers with the latest SparklyR version.

* Additional bugfixes, stability, and notebook improvements.

2018-04-17 10:42

v0.12

New functionality:

* MMLSpark Serving: a RESTful computation engine built on Spark
  streaming.  See `docs/mmlspark-serving.md` for details.

* New LightGBM Binary Classification and Regression learners and
  infrastructure with a Python notebook for examples.

* MMLSpark Clients: a general-purpose, distributed, and fault tolerant
  HTTP Library usable from Spark, Pyspark, and SparklyR.  See
  `docs/http.md`.

* Add `MinibatchTransformer` and `FlattenBatch` to enable efficient,
  buffered, minibatch processing in Spark.

* Added Python wrappers and a notebook example for the
  `TuneHyperparameters` module, demonstrating parallel distributed
  hyperparameter tuning through randomized grid search.

* Add a `MultiNGram` transformer for efficiently computing variable
  length n-grams.

* Added DataType parameter for building models that are parameterized by
  Spark data types.

Updates:

* Update per-instance statistics module so it works for any Spark ML
  estimators.

* Update CNTK to version 2.4.

* Updated Spark to version v2.2.1 (the following release is likely to be
  based on Spark 2.3).

* Also updated SBT and JVM.

* Refactored readers directory into `io` directory

Improvements:

* Fix the Conda installation in our Docker image, resolving issues with
  importing `numpy`.

* Fix a regression in R wrappers with the latest SparklyR version.

* Additional bugfixes, stability, and notebook improvements.

2018-04-17 10:42

2018-01-31 03:47

2018-01-31 03:47

mmlspark-v0.10.9

2018-01-26 05:01

2018-01-26 05:01

v0.10

New functionality:

* We now provide initial support for training on a GPU VM, and an ARM
  template to deploy an HDI Cluster with an associated GPU machine.  See
  `docs/gpu-setup.md` for instructions on setting this up.

* New auto-generated R wrappers for estimators and transformers.  To
  import them into R, you can use devtools to import from the uploaded
  zip file.  Tests and sample notebooks to come.

* A new `RenameColumn` transformer for renaming columns within a
  pipeline.

New notebooks:

* Notebook 104: An experiment to demonstrate regression models to
  predict automobile prices.  This notebook demonstrates the use of
  `Pipeline` stages, `CleanMissingData`, and
  `ComputePerInstanceStatistics`.

* Notebook 105: Demonstrates `DataConversion` to make some columns Categorical.

* There us a 401 notebook in `notebooks/gpu` which demonstrates CNTK
  training when using a GPU VM.  (It is not shown with the rest of the
  notebooks yet.)

Updates:

* Updated to use CNTK 2.2.  Note that this version of CNTK depends on
  libpng12 and libjasper1 -- which are included in our docker images.
  (This should get resolved in the upcoming CNTK 2.3 release.)

Improvements:

* Local builds will always use a "0.0" version instead of a version
  based on the git repository.  This should simplify the build process
  for developers and avoid hard-to-resolve update issues.

* The `TextPreprocessor` transformer can be used to find and replace all
  key value pairs in an input map.

* Fixed a regression in the image reader where zip files with images no
  longer displayed the full path to the image inside a zip file.

* Additional minor bug and stability fixes.

2017-11-15 13:59

v0.10

New functionality:

* We now provide initial support for training on a GPU VM, and an ARM
  template to deploy an HDI Cluster with an associated GPU machine.  See
  `docs/gpu-setup.md` for instructions on setting this up.

* New auto-generated R wrappers for estimators and transformers.  To
  import them into R, you can use devtools to import from the uploaded
  zip file.  Tests and sample notebooks to come.

* A new `RenameColumn` transformer for renaming columns within a
  pipeline.

New notebooks:

* Notebook 104: An experiment to demonstrate regression models to
  predict automobile prices.  This notebook demonstrates the use of
  `Pipeline` stages, `CleanMissingData`, and
  `ComputePerInstanceStatistics`.

* Notebook 105: Demonstrates `DataConversion` to make some columns Categorical.

* There us a 401 notebook in `notebooks/gpu` which demonstrates CNTK
  training when using a GPU VM.  (It is not shown with the rest of the
  notebooks yet.)

Updates:

* Updated to use CNTK 2.2.  Note that this version of CNTK depends on
  libpng12 and libjasper1 -- which are included in our docker images.
  (This should get resolved in the upcoming CNTK 2.3 release.)

Improvements:

* Local builds will always use a "0.0" version instead of a version
  based on the git repository.  This should simplify the build process
  for developers and avoid hard-to-resolve update issues.

* The `TextPreprocessor` transformer can be used to find and replace all
  key value pairs in an input map.

* Fixed a regression in the image reader where zip files with images no
  longer displayed the full path to the image inside a zip file.

* Additional minor bug and stability fixes.

2017-11-15 13:59

mmlspark-v0.9.9

2017-08-31 02:22

2017-08-31 02:22

2017-10-13 06:19

2017-10-13 06:19

mmlspark-v0.8.9

2017-08-31 02:22

2017-08-31 02:22

下载

请输入验证码，防止盗链导致资源被占用

1 2 3 4

Scala

1

https://gitee.com/hgkdt/MMLSpark.git

git@gitee.com:hgkdt/MMLSpark.git

hgkdt

MMLSpark

MMLSpark

深圳市奥思网络科技有限公司版权所有

Git 命令学习

CopyCat 代码克隆检测

APP与插件下载

Gitee 封面人物

Gitee 公益计划

Gitee 持续集成

在线自助服务

技术交流QQ群

微信服务号

client#oschina.cn

企业版在线使用：400-606-0201

专业版私有部署：

开放原子开源基金会

合作代码托管平台

违法和不良信息举报中心

粤ICP备12009483号

简体 / 繁體 / English

点此查找更多帮助

搜索帮助

Git 命令在线学习如何在 Gitee 导入 GitHub 仓库

Git 仓库基础操作

企业版和社区版功能对比

SSH 公钥设置

如何处理代码冲突

仓库体积过大，如何减小？

如何找回被删除的仓库数据

Gitee 产品配额说明

GitHub仓库快速导入Gitee及同步更新

什么是 Release（发行版）

将 PHP 项目自动发布到 packagist.org

评论

仓库举报

回到顶部

登录提示

该操作需登录 Gitee 帐号，请先登录后再操作。

立即登录

没有帐号，去注册