dataPrepare.ipynb · liuqiang123456789/unet

加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

文件

克隆/下载

dataPrepare.ipynb 3.74 KB

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\SoftWare\\Anaconda2\\envs\\python3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from data import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# data augmentation \n",
    "\n",
    "In deep learning tasks, a lot of data is need to train DNN model, when the dataset is not big enough, data augmentation should be applied.\n",
    "\n",
    "keras.preprocessing.image.ImageDataGenerator is a data generator, which can feed the DNN with data like : (data,label), it can also do data augmentation at the same time.\n",
    "\n",
    "It is very convenient for us to use keras.preprocessing.image.ImageDataGenerator to do data augmentation by implement image rotation, shift, rescale and so on... see [keras documentation](https://keras.io/preprocessing/image/) for detail.\n",
    "\n",
    "For image segmentation tasks, the image and mask must be transformed **together!!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## define your data generator\n",
    "\n",
    "If you want to visualize your data augmentation result, set save_to_dir = your path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#if you don't want to do data augmentation, set data_gen_args as an empty dict.\n",
    "#data_gen_args = dict()\n",
    "\n",
    "data_gen_args = dict(rotation_range=0.2,\n",
    "                    width_shift_range=0.05,\n",
    "                    height_shift_range=0.05,\n",
    "                    shear_range=0.05,\n",
    "                    zoom_range=0.05,\n",
    "                    horizontal_flip=True,\n",
    "                    fill_mode='nearest')\n",
    "myGenerator = trainGenerator(20,'data/membrane/train','image','label',data_gen_args,save_to_dir = \"data/membrane/train/aug\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## visualize your data augmentation result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#you will see 60 transformed images and their masks in data/membrane/train/aug\n",
    "num_batch = 3\n",
    "for i,batch in enumerate(myGenerator):\n",
    "    if(i >= num_batch):\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## create .npy data\n",
    "\n",
    "If your computer has enough memory, you can create npy files containing all your images and masks, and feed your DNN with them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_arr,mask_arr = geneTrainNpy(\"data/membrane/train/aug/\",\"data/membrane/train/aug/\")\n",
    "#np.save(\"data/image_arr.npy\",image_arr)\n",
    "#np.save(\"data/mask_arr.npy\",mask_arr)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}

一键复制编辑原始数据按行查看历史

提交于 2018-06-22 10:37 . Update dataPrepare.ipynb

马建仓 AI 助手

尝试更多

代码解读

代码找茬

代码优化

https://gitee.com/liuqiang123456789/unet.git

git@gitee.com:liuqiang123456789/unet.git

liuqiang123456789

unet

master

liuqiang123456789/unet

简介

发行版

贡献者

近期动态

liuqiang123456789/unet .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

liuqiang123456789/unet