第一章.md · 杨雪元/machine_learning

---
typora-copy-images-to: picture
typora-root-url: ./
---
### 总结

k近邻比较慢，不适合多维特征或者稀疏矩阵

但容易理解，通常不需要多调整就能获得不错的性能

线性回归的话对于一维不太适用，模型比较简单，但对于多维来说会有较大的提升，模型复杂，相应的也会加剧过拟合的可能性。

加入正则抑制过拟合的用于回归问题的有ridge和lasso

用于分类的线性回归是同样的方程，但设置了阈值

最常见的两种线性分类算法是 Logistic 回归（logistic regression）和线性支持向量机（linear

support vector machine，线性 SVM），前者在 linear_model.LogisticRegression 中实现，

后者在 svm.LinearSVC（SVC 代表支持向量分类器）中实现。虽然 LogisticRegression

的名字中含有回归（regression），但它是一种分类算法，并不是回归算法，不应与

LinearRegression 混淆

线性模型的训练速度非常快，预测速度也很快。这种模型可以推广到非常大的数据集，对

稀疏数据也很有效。如果你的数据包含数十万甚至上百万个样本，你可能需要研究如何使

用 LogisticRegression 和 Ridge 模型的 solver='sag' 选项，在处理大型数据时，这一选项

比默认值要更快。其他选项还有 SGDClassifier 类和 SGDRegressor 类，它们对本节介绍的

线性模型实现了可扩展性更强的版本。

线性模型的另一个优点在于，利用我们之间见过的用于回归和分类的公式，理解如何进

行预测是相对比较容易的。不幸的是，往往并不完全清楚系数为什么是这样的。如果你

的数据集中包含高度相关的特征，这一问题尤为突出。在这种情况下，可能很难对系数

做出解释。如果特征数量大于样本数量，线性模型的表现通常都很好。它也常用于非常大的数据集，

只是因为训练其他模型并不可行。但在更低维的空间中，其他模型的泛化性能可能更好。

线性模型适用于高维数据。

朴素贝叶斯分类器是与上一节介绍的线性模型非常相似的一种分类器，但它的训练速度往

往更快。这种高效率所付出的代价是，朴素贝叶斯模型的泛化能力要比线性分类器（如

LogisticRegression 和 LinearSVC）稍差。

朴素贝叶斯模型如此高效的原因在于，它通过**单独查看每个特征来学习**参数，并从每

个特征中收集简单的类别统计数据。scikit-learn 中实现了三种朴素贝叶斯分类器：

GaussianNB、BernoulliNB 和 MultinomialNB。GaussianNB 可 应 用 于 任 意 连 续 数 据， 而

BernoulliNB 假定输入数据为二分类数据，MultinomialNB 假定输入数据为计数数据（即每

个特征代表某个对象的整数计数，比如一个单词在句子里出现的次数）。BernoulliNB 和

MultinomialNB 主要用于文本数据分类。

![image-20211110141515762](/picture/image-20211110141515762.png)

2.3.7 节会介绍几个线性模型不适用的例子。

![image-20211106164844790](C:\Users\Yangxueyuan1\Desktop\introduction_to_ml_with_python\picture\image-20211106164844790.png)

~~~
from scipy import sparse
# Create a 2D NumPy array with a diagonal of ones, and zeros everywhere else
eye = np.eye(4)
print("NumPy array:\n", eye)
# Convert the NumPy array to a SciPy sparse matrix in CSR format
# Only the nonzero entries are stored
sparse_matrix = sparse.csr_matrix(eye)
print("\nSciPy sparse CSR matrix:\n", sparse_matrix)
~~~

![image-20211106165048688](/picture/image-20211106165048688.png)

~~~
data = np.ones(4)
row_indices = np.arange(4)
col_indices = np.arange(4)
eye_coo = sparse.coo_matrix((data, (row_indices, col_indices)))
print("COO representation:\n", eye_coo)
~~~

这里是coco模式

![image-20211106165001551](/picture/image-20211106165001551.png)

![image-20211106165017205](/picture/image-20211106165017205.png)

![image-20211106165113985](/picture/image-20211106165113985.png)

~~~
import pandas as pd

# create a simple dataset of people
data = {'Name': ["John", "Anna", "Peter", "Linda"],
        'Location' : ["New York", "Paris", "Berlin", "London"],
        'Age' : [24, 13, 53, 33]
       }

data_pandas = pd.DataFrame(data)
# IPython.display allows "pretty printing" of dataframes
# in the Jupyter notebook
display(data_pandas)
~~~

![image-20211106165137706](/picture/image-20211106165137706.png)

![image-20211106165200620](/picture/image-20211106165200620.png)

![image-20211106165728889](/picture/image-20211106165728889.png)

![image-20211106165832692](/picture/image-20211106165832692.png)

![image-20211106165908797](/picture/image-20211106165908797.png)

![image-20211106170040874](/picture/image-20211106170040874.png)

![image-20211106170056212](/picture/image-20211106170056212.png)

![image-20211106170112589](/picture/image-20211106170112589.png)

![image-20211106170144391](/picture/image-20211106170144391.png)

![image-20211106170306032](/picture/image-20211106170306032.png)

![image-20211106170410123](/picture/image-20211106170410123.png)

![image-20211106170428265](/picture/image-20211106170428265.png)

![image-20211106170522836](/picture/image-20211106170522836.png)

![image-20211106170644091](/picture/image-20211106170644091.png)

~~~
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
    iris_dataset['data'], iris_dataset['target'], random_state=0)
~~~

![image-20211106170929434](/picture/image-20211106170929434.png)

![image-20211106170956964](/picture/image-20211106170956964.png)

![image-20211106171057152](/picture/image-20211106171057152.png)

~~~
# create dataframe from data in X_train
# label the columns using the strings in iris_dataset.feature_names
iris_dataframe = pd.DataFrame(X_train, columns=iris_dataset.feature_names)
# create a scatter matrix from the dataframe, color by y_train
pd.plotting.scatter_matrix(iris_dataframe, c=y_train, figsize=(15, 15),
                           marker='o', hist_kwds={'bins': 20}, s=60,
                           alpha=.8, cmap=mglearn.cm3)
~~~

>散点图矩阵图，可以两两发现特征之间的联系
>
>第一个参数 dataframe：iris_dataframe 按行取数据
>第二个参数 c=c=y_train 颜色，用不同着色度区分不同种类
>三：figsize=(15,15) 图像区域大小，英寸为单位
>四：marker=‘0’ 点的形状，0是圆，1是￥
>五： hist_kwds={‘bins’:50} 对角线上直方图的参数元组
>六：s=60 描出点的大小
>七：alpha=.8 图像透明度，一般取(0,1]
>八：cmap=mglearn.cm3 mylearn实用函数库，主要对图进行一些美化等私有功能，可见https://github.com/YifengChu/introduction_to_ml_with_python

![image](/picture/output.png)

我们可以看到通过两两特征向量的组合可以很好的区分这三个类别

~~~
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=1)
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=1)
~~~

![image-20211107094146508](/picture/image-20211107094146508.png)

![image-20211107094230052](/picture/image-20211107094230052.png)

![image-20211107094510073](/picture/image-20211107094510073.png)

![image-20211107102044584](/picture/image-20211107102044584.png)

~~~
X_new = np.array([[5, 2.9, 1, 0.2]])
print("X_new.shape:", X_new.shape)
~~~

~~~
prediction = knn.predict(X_new)
print("Prediction:", prediction)
print("Predicted target name:",
       iris_dataset['target_names'][prediction])
~~~

![image-20211107104159327](/picture/image-20211107104159327.png)

![image-20211107104221258](/picture/image-20211107104221258.png)

~~~
y_pred = knn.predict(X_test)
print("Test set predictions:\n", y_pred)
~~~

~~~
print("Test set score: {:.2f}".format(np.mean(y_pred == y_test)))
~~~

np.mean求取平均值

有一个集成的方法来进行判断

~~~
print("Test set score: {:.2f}".format(knn.score(X_test, y_test)))
~~~

![image-20211107104527426](/picture/image-20211107104527426.png)

![image-20211107104615100](/picture/image-20211107104615100.png)

~~~
X_train, X_test, y_train, y_test = train_test_split(
    iris_dataset['data'], iris_dataset['target'], random_state=0)

knn = KNeighborsClassifier(n_neighbors=1)
knn.fit(X_train, y_train)

print("Test set score: {:.2f}".format(knn.score(X_test, y_test)))
~~~

![image-20211108184222032](/picture/image-20211108184222032.png)

#### 第二章

![image-20211108184401744](picture\image-20211108184401744.png)

![image-20211108184510833](/picture/image-20211108184510833.png)

![image-20211108190957611](/picture/image-20211108190957611.png)

![image-20211108191046629](/picture/image-20211108191046629.png)

![image-20211108191129928](/picture/image-20211108191129928.png)

![image-20211108191153014](/picture/image-20211108191153014.png)

![image-20211108191239209](/picture/image-20211108191239209.png)

![image-20211108191353183](/picture/image-20211108191353183.png)

![image-20211108191403325](/picture/image-20211108191403325.png)

![image-20211108191435254](/picture/image-20211108191435254.png)

![image-20211108191722698](/picture/image-20211108191722698.png)

![image-20211108191810993](/picture/image-20211108191810993.png)

![image-20211108192449987](/picture/image-20211108192449987.png)

~~~
# generate dataset
X, y = mglearn.datasets.make_forge()
# plot dataset
mglearn.discrete_scatter(X[:, 0], X[:, 1], y)
plt.legend(["Class 0", "Class 1"], loc=4)
plt.xlabel("First feature")
plt.ylabel("Second feature")
print("X.shape:", X.shape)
~~~

![image-20211108192539509](/picture/image-20211108192539509.png)

![image-20211108192557134](/picture/image-20211108192557134.png)

![image-20211108192611060](/picture/image-20211108192611060.png)

###### 分类使用forge

###### 回归使用wave

~~~
X, y = mglearn.datasets.make_wave(n_samples=40)
plt.plot(X, y, 'o')
plt.ylim(-3, 3)
plt.xlabel("Feature")
plt.ylabel("Target")
~~~

![image-20211108192927957](/picture/image-20211108192927957.png)

![image-20211108192954871](/picture/image-20211108192954871.png)

![image-20211108194353117](/picture/image-20211108194353117.png)

![image-20211108194403682](/picture/image-20211108194403682.png)

~~~
from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
print("cancer.keys():\n", cancer.keys())
~~~

![image-20211108194513851](/picture/image-20211108194513851.png)

![image-20211108194700375](/picture/image-20211108194700375.png)

~~~
print("Sample counts per class:\n",
      {n: v for n, v in zip(cancer.target_names, np.bincount(cancer.target))})
      # zip会打包为元组的形式
~~~

它大致说bin的数量比x中的最大值大1，每个bin给出了它的索引值在x中出现的次数。下面，我举个例子让大家更好的理解一下：

```python
# 我们可以看到x中最大的数为7，因此bin的数量为8，那么它的索引值为0->7
x = np.array([0, 1, 1, 3, 2, 1, 7])
# 索引0出现了1次，索引1出现了3次......索引5出现了0次......
np.bincount(x)
#因此，输出结果为：array([1, 3, 1, 1, 0, 0, 0, 1])
```

这里的target为0 1

![image-20211108195251597](/picture/image-20211108195251597.png)

![image-20211108195313917](/picture/image-20211108195313917.png)

~~~
from sklearn.datasets import load_boston
boston = load_boston()
print("Data shape:", boston.data.shape)
~~~

![image-20211108195445365](/picture/image-20211108195445365.png)

![image-20211108195942193](/picture/image-20211108195942193.png)

![image-20211108195956071](/picture/image-20211108195956071.png)

![image-20211108200103463](/picture/image-20211108200103463.png)

![image-20211108200115602](/picture/image-20211108200115602.png)

![image-20211108200221378](/picture/image-20211108200221378.png)

![image-20211108200333991](/picture/image-20211108200333991.png)

~~~
from sklearn.model_selection import train_test_split
X, y = mglearn.datasets.make_forge()
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
from sklearn.neighbors import KNeighborsClassifier
clf = KNeighborsClassifier(n_neighbors=3)
clf.fit(X_train, y_train)
~~~

![image-20211108200458490](/picture/image-20211108200458490.png)

![image-20211108200610083](/picture/image-20211108200610083.png)

![image-20211108200640744](/picture/image-20211108200640744.png)

~~~
fig, axes = plt.subplots(1, 3, figsize=(10, 3))
print(axes.all)
# ax其实就是画图的位置，无需过多关注
for n_neighbors, ax in zip([1, 3, 9], axes):
    # the fit method returns the object self, so we can instantiate
    # and fit in one line
    clf = KNeighborsClassifier(n_neighbors=n_neighbors).fit(X, y)
    mglearn.plots.plot_2d_separator(clf, X, fill=True, eps=0.5, ax=ax, alpha=.4)
    mglearn.discrete_scatter(X[:, 0], X[:, 1], y, ax=ax)
    ax.set_title("{} neighbor(s)".format(n_neighbors))
    ax.set_xlabel("feature 0")
    ax.set_ylabel("feature 1")
axes[0].legend(loc=3)
~~~

![image-20211108201221006](/picture/image-20211108201221006.png)

![image-20211108201231520](/picture/image-20211108201231520.png)

![image-20211108201608536](/picture/image-20211108201608536.png)

![image-20211108201811426](/picture/image-20211108201811426.png)

![image-20211108202204541](/picture/image-20211108202204541.png)

![image-20211108202316751](/picture/image-20211108202316751.png)

![image-20211108202333061](/picture/image-20211108202333061.png)

![image-20211108202353416](/picture/image-20211108202353416.png)

![image-20211108202442680](/picture/image-20211108202442680.png)

~~~
from sklearn.neighbors import KNeighborsRegressor

X, y = mglearn.datasets.make_wave(n_samples=60)

# split the wave dataset into a training and a test set
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)

# instantiate the model and set the number of neighbors to consider to 3
reg = KNeighborsRegressor(n_neighbors=7)
# fit the model using the training data and training targets
reg.fit(X_train, y_train)
print("Test set predictions:\n", reg.predict(X_test))
print("Test set R^2: {:.2f}".format(reg.score(X_test, y_test)))
~~~

![image-20211108202603453](/picture/image-20211108202603453.png)

![image-20211108202759481](/picture/image-20211108202759481.png)

~~~
fig, axes = plt.subplots(1, 3, figsize=(15, 4))
# create 1,000 data points, evenly spaced between -3 and 3
line = np.linspace(-3, 3, 1000).reshape(-1, 1)
for n_neighbors, ax in zip([1, 3, 9], axes):
    # make predictions using 1, 3, or 9 neighbors
    reg = KNeighborsRegressor(n_neighbors=n_neighbors)
    reg.fit(X_train, y_train)
    ax.plot(line, reg.predict(line))
    ax.plot(X_train, y_train, '^', c=mglearn.cm2(0), markersize=8)
    ax.plot(X_test, y_test, 'v', c=mglearn.cm2(1), markersize=8)

ax.set_title(
        "{} neighbor(s)\n train score: {:.2f} test score: {:.2f}".format(
            n_neighbors, reg.score(X_train, y_train),
            reg.score(X_test, y_test)))
    ax.set_xlabel("Feature")
    ax.set_ylabel("Target")
axes[0].legend(["Model predictions", "Training data/target",
                "Test data/target"], loc="best")
                # 中间的那一条线是line的先，我们可以看到它越来越平滑
~~~

![image-20211108202944008](/picture/image-20211108202944008.png)

![image-20211108203008390](/picture/image-20211108203008390.png)

![image-20211108203117742](/picture/image-20211108203117742.png)

k近邻缺点  速度慢，不适合有很多特征的，特别是很多特征取值为0 的

杨雪元/machine_learning

简介

发行版

贡献者

近期动态

杨雪元/machine_learning .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

杨雪元/machine_learning