代码拉取完成,页面将自动刷新
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>大数据存储数据优化</title>
<link href="/2019/11/21/%E6%95%B0%E6%8D%AE%E5%BA%93/%E5%A4%A7%E6%95%B0%E6%8D%AE%E5%AD%98%E5%82%A8%E4%BC%98%E5%8C%96/"/>
<url>/2019/11/21/%E6%95%B0%E6%8D%AE%E5%BA%93/%E5%A4%A7%E6%95%B0%E6%8D%AE%E5%AD%98%E5%82%A8%E4%BC%98%E5%8C%96/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="大数据存储数据优化"><a href="#大数据存储数据优化" class="headerlink" title="大数据存储数据优化"></a>大数据存储数据优化</h2><p>接着前面搭的项目继续今天得内容,现在项目结构如下:</p><p><img alt="enter image description here" data-src="/img/DSO/9B9CCB21-702C-4823-A837-326A857D10EB.png" class="lazyload"></p><p>新增了网关服务和集成了SpringData JPA组件。今天就先暂用Order服务来测试数据;</p><h4 id="准备环境"><a href="#准备环境" class="headerlink" title="准备环境"></a>准备环境</h4><p>服务名称:orderService(订单服务),zuulService(网关服务)</p><h5 id="orderService"><a href="#orderService" class="headerlink" title="orderService"></a>orderService</h5><p>pom.xml:</p><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependencies</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.cloud<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-cloud-starter-netflix-eureka-client<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!--springboot 热部署--></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-devtools<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-jdbc<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>com.alibaba<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>druid<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>1.0.18<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>mysql<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>mysql-connector-java<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!--测试环境--></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-test<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>test<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!--JPA--></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-data-jpa<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="comment"><!-- lombok --></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.projectlombok<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>lombok<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>1.16.10<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-test<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>test<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">exclusions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">exclusion</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.junit.vintage<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>junit-vintage-engine<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">exclusion</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">exclusions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-web<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>2.2.1.RELEASE<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>compile<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>junit<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>junit<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.apache.curator<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>curator-framework<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">version</span>></span>2.6.0<span class="tag"></<span class="name">version</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependencies</span>></span></span><br></pre></td></tr></table></figure><p>其中 springboot 热部署 和 lombok 可以节约很多调试和开发时间,强烈推荐!!!</p><p>application.properties配置如下:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">server.port=8082</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 应用名称</span></span><br><span class="line"><span class="string">spring.application.name=order</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># eureka服务端地址</span></span><br><span class="line"><span class="string">eureka.client.service-url.defaultZone=http://localhost:8761/eureka/</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 配置默认数据源,并使用 阿里连接池</span></span><br><span class="line"><span class="string">spring.datasource.type=com.alibaba.druid.pool.DruidDataSource</span></span><br><span class="line"><span class="comment"># 注意 rewriteBatchedStatements=true 这个参数是开启批处理 很重要!!! 测试1W条数据没他许需要4分钟 开启只需要762ms 需要搭配JdbcTemplate使用</span></span><br><span class="line"><span class="string">spring.datasource.url=jdbc:mysql://localhost:3306/spring_data?serverTimezone=UTC&rewriteBatchedStatements=true</span></span><br><span class="line"><span class="string">spring.datasource.username=root</span></span><br><span class="line"><span class="string">spring.datasource.password=root</span></span><br><span class="line"><span class="string">spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver</span></span><br><span class="line"><span class="string">spring.datasource.max-active=20</span></span><br><span class="line"><span class="string">spring.datasource.max-idle=8</span></span><br><span class="line"><span class="string">spring.datasource.min-idle=8</span></span><br><span class="line"><span class="string">spring.datasource.initial-size=10</span> </span><br><span class="line"></span><br><span class="line"><span class="comment">######JPA 配置#####</span></span><br><span class="line"><span class="comment"># 声明数据库</span></span><br><span class="line"><span class="string">spring.jpa.database=mysql</span></span><br><span class="line"><span class="comment"># 是否显示SQL语句</span></span><br><span class="line"><span class="string">spring.jpa.show-sql=true</span></span><br><span class="line"><span class="comment"># Hibernate 自动DDL 操作</span></span><br><span class="line"><span class="comment"># create 每次加载hibernate时都会删除上一次的生成的表</span></span><br><span class="line"><span class="comment"># create-drop 每次加载hibernate时根据model类生成表,但是sessionFactory一关闭,表就自动删除</span></span><br><span class="line"><span class="comment"># update 最常用的属性,第一次加载hibernate时根据model类会自动建立起表的结构(前提是先建立好数据库)</span></span><br><span class="line"><span class="string">spring.jpa.hibernate.ddl-auto=update</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#配置方言</span></span><br><span class="line"><span class="string">spring.jpa.database-platform=org.hibernate.dialect.MySQL5Dialect</span></span><br><span class="line"><span class="string">spring.jpa.properties.hibernate.generate_statistics=true</span></span><br><span class="line"><span class="string">spring.jpa.properties.hibernate.jdbc.batch_size=200</span></span><br><span class="line"><span class="string">spring.jpa.properties.hibernate.order_inserts=true</span></span><br></pre></td></tr></table></figure><p>注意:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 这是老的JDBC驱动</span></span><br><span class="line"><span class="string">spring.datasource.driverClassName</span> <span class="string">=</span> <span class="string">com.mysql.jdbc.Driver</span></span><br><span class="line"><span class="comment"># 这是新的 不更新会启动会报错</span></span><br><span class="line"><span class="string">spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver</span></span><br></pre></td></tr></table></figure><p>开启批处理:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">spring.datasource.url=jdbc:mysql://localhost:3306/spring_data?serverTimezone=UTC&rewriteBatchedStatements=true</span></span><br></pre></td></tr></table></figure><p>配置参数rewriteBatchedStatements 为 true;具体什么用后面会介绍。</p><h5 id="zuulService"><a href="#zuulService" class="headerlink" title="zuulService"></a>zuulService</h5><p>pom.xml:</p><figure class="highlight xml"><table><tr><td class="code"><pre><span class="line"><span class="tag"><<span class="name">dependencies</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.cloud<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-cloud-starter-netflix-zuul<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.cloud<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-cloud-starter-netflix-eureka-client<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.cloud<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-cloud-starter-ribbon<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"></span><br><span class="line"> <span class="tag"><<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.springframework.boot<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>spring-boot-starter-test<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">scope</span>></span>test<span class="tag"></<span class="name">scope</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">exclusions</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">exclusion</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">groupId</span>></span>org.junit.vintage<span class="tag"></<span class="name">groupId</span>></span></span><br><span class="line"> <span class="tag"><<span class="name">artifactId</span>></span>junit-vintage-engine<span class="tag"></<span class="name">artifactId</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">exclusion</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">exclusions</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependency</span>></span></span><br><span class="line"> <span class="tag"></<span class="name">dependencies</span>></span></span><br></pre></td></tr></table></figure><p>application.properties配置如下:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">server.port=8083</span></span><br><span class="line"><span class="string">spring.application.name=zuul</span></span><br><span class="line"><span class="string">eureka.client.service-url.defaultZone=http://localhost:8761/eureka/</span></span><br><span class="line"><span class="comment"># 路由前缀</span></span><br><span class="line"><span class="string">zuul.routes.zuul.path=/hello/**</span></span><br><span class="line"><span class="comment"># 通过id 指定访问得服务名称(这里指定的是order服务)</span></span><br><span class="line"><span class="string">zuul.routes.zuul.service-id=order</span> </span><br><span class="line"><span class="string">zuul.host.connect-timeout-millis=50000</span></span><br><span class="line"><span class="string">zuul.host.socket-timeout-millis=60000</span></span><br><span class="line"><span class="string">ribbon.ConnectTimeout=3000</span></span><br><span class="line"><span class="string">ribbon.ReadTimeout=10000</span></span><br></pre></td></tr></table></figure><p>注意:<br>zuul也需要当成服务注册eureka,这样才能拿到所有服务的信息,做相应的处理。关于zuul的作用可以点击这:<a href="https://www.jianshu.com/p/d49eb9b26351" target="_blank" rel="noopener">传送门</a></p><h4 id="数据准备"><a href="#数据准备" class="headerlink" title="数据准备"></a>数据准备</h4><h5 id="Entity"><a href="#Entity" class="headerlink" title="Entity"></a>Entity</h5><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> com.example.order.base.BaseEntity;</span><br><span class="line"><span class="keyword">import</span> lombok.Data;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> javax.persistence.Column;</span><br><span class="line"><span class="keyword">import</span> javax.persistence.Entity;</span><br><span class="line"><span class="keyword">import</span> javax.persistence.Table;</span><br><span class="line"><span class="keyword">import</span> javax.validation.constraints.NotBlank;</span><br><span class="line"><span class="keyword">import</span> java.math.BigDecimal;</span><br><span class="line"></span><br><span class="line"><span class="comment">//lombok注解 自动生成get set</span></span><br><span class="line"><span class="meta">@Data</span></span><br><span class="line"><span class="meta">@Entity</span></span><br><span class="line"><span class="meta">@Table</span>(name = <span class="string">"order_info"</span>)</span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">Order</span> <span class="keyword">extends</span> <span class="title">BaseEntity</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@NotBlank</span></span><br><span class="line"> <span class="meta">@Column</span>(columnDefinition = <span class="string">"varchar(18) default ' ' comment '订单名称' "</span>)</span><br><span class="line"> <span class="keyword">private</span> String name;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@NotBlank</span></span><br><span class="line"> <span class="meta">@Column</span>(columnDefinition = <span class="string">"varchar(18) default ' ' comment '订单编码' "</span>)</span><br><span class="line"> <span class="keyword">private</span> String code;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Column</span>(columnDefinition = <span class="string">"decimal(18,2) default 0 comment '订单价格' "</span>)</span><br><span class="line"> <span class="keyword">private</span> BigDecimal price;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@NotBlank</span></span><br><span class="line"> <span class="meta">@Column</span>(columnDefinition = <span class="string">"varchar(18) default ' ' comment '会员Id' "</span>)</span><br><span class="line"> <span class="keyword">private</span> String userId;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Column</span>(columnDefinition = <span class="string">"varchar(128) default '' comment '备注' "</span>)</span><br><span class="line"> <span class="keyword">private</span> String remark;</span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>Repository:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> com.example.order.dao.mysql.Order;</span><br><span class="line"><span class="keyword">import</span> org.springframework.data.repository.CrudRepository;</span><br><span class="line"><span class="keyword">import</span> org.springframework.stereotype.Repository;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Repository</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">interface</span> <span class="title">OrderRepository</span> <span class="keyword">extends</span> <span class="title">CrudRepository</span><<span class="title">Order</span>, <span class="title">String</span>> </span>{</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>Service:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> com.example.order.base.SnowFlake;</span><br><span class="line"><span class="keyword">import</span> com.example.order.dao.mysql.Order;</span><br><span class="line"><span class="keyword">import</span> com.example.order.repository.OrderRepository;</span><br><span class="line"><span class="keyword">import</span> com.example.order.servcie.OrderService;</span><br><span class="line"><span class="keyword">import</span> com.example.order.vo.req.OrderSaveReqVo;</span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.annotation.Autowired;</span><br><span class="line"><span class="keyword">import</span> org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;</span><br><span class="line"><span class="keyword">import</span> org.springframework.stereotype.Service;</span><br><span class="line"><span class="keyword">import</span> org.springframework.transaction.annotation.Transactional;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.sql.Timestamp;</span><br><span class="line"><span class="keyword">import</span> java.util.ArrayList;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">OrderServiceImpl</span> <span class="keyword">implements</span> <span class="title">OrderService</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> OrderRepository orderRepository;</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> NamedParameterJdbcTemplate namedParameterJdbcTemplate;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="meta">@Transactional</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> String <span class="title">save</span><span class="params">(OrderSaveReqVo reqVo)</span> </span>{</span><br><span class="line"> <span class="comment">//雪花算法分布式下生成唯一id</span></span><br><span class="line"> SnowFlake snowFlake = <span class="keyword">new</span> SnowFlake(<span class="number">1</span>, <span class="number">1</span>);</span><br><span class="line"></span><br><span class="line"> ArrayList<Order> orders = <span class="keyword">new</span> ArrayList<Order>();</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i <= <span class="number">1000</span>; i++) {</span><br><span class="line"></span><br><span class="line"> Order order = <span class="keyword">new</span> Order();</span><br><span class="line"> order.setCode(reqVo.getCode());</span><br><span class="line"> order.setName(reqVo.getName());</span><br><span class="line"> order.setRemark(reqVo.getRemark());</span><br><span class="line"> order.setPrice(reqVo.getPrice());</span><br><span class="line"> order.setUserId(reqVo.getUserId());</span><br><span class="line"> order.setId(snowFlake.nextId());</span><br><span class="line"> order.setCreateTimestamp(<span class="keyword">new</span> Timestamp(System.currentTimeMillis()));</span><br><span class="line"></span><br><span class="line"> orders.add(order);</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> <span class="comment">/* SqlParameterSource[] beanSources = SqlParameterSourceUtils.createBatch(orders.toArray());</span></span><br><span class="line"><span class="comment"> String sql = "insert into order_info (id,code,name,remark,price,user_id,create_timestamp) values (:id,:code,:name,:remark,:price,:userId,:createTimestamp)";</span></span><br><span class="line"><span class="comment"> namedParameterJdbcTemplate.batchUpdate(sql, beanSources);*/</span></span><br><span class="line"> orderRepository.saveAll(orders);</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"成功"</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>一个简单的保存操作我们来一个一个测试保存数据的性能如何;</p><p>方案一:</p><p>我们将数据源参数:”rewriteBatchedStatements=true”删除,1K条数据测试保存时间:</p><p><img alt="enter image description here" data-src="/img/DSO/5325AD12-1BDC-4246-9BC1-A44202938829.png" class="lazyload"></p><p>当时我看完都有点懵,1K条简单的数据保存了38s…..要被祭天。<br>分析:</p><blockquote><p>代码层次:一个简单的for循环,一个简单的保存,排除。极大可能就只有JPA saveAll会耗费很长时间;面向百度编程,可以使用JdbcTemplate批量保存。修改代码如下:</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">import</span> com.example.order.base.SnowFlake;</span><br><span class="line"><span class="keyword">import</span> com.example.order.dao.mysql.Order;</span><br><span class="line"><span class="keyword">import</span> com.example.order.repository.OrderRepository;</span><br><span class="line"><span class="keyword">import</span> com.example.order.servcie.OrderService;</span><br><span class="line"><span class="keyword">import</span> com.example.order.vo.req.OrderSaveReqVo;</span><br><span class="line"><span class="keyword">import</span> org.springframework.beans.factory.annotation.Autowired;</span><br><span class="line"><span class="keyword">import</span> org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate;</span><br><span class="line"><span class="keyword">import</span> org.springframework.jdbc.core.namedparam.SqlParameterSource;</span><br><span class="line"><span class="keyword">import</span> org.springframework.jdbc.core.namedparam.SqlParameterSourceUtils;</span><br><span class="line"><span class="keyword">import</span> org.springframework.stereotype.Service;</span><br><span class="line"><span class="keyword">import</span> org.springframework.transaction.annotation.Transactional;</span><br><span class="line"></span><br><span class="line"><span class="keyword">import</span> java.sql.Timestamp;</span><br><span class="line"><span class="keyword">import</span> java.util.ArrayList;</span><br><span class="line"></span><br><span class="line"><span class="meta">@Service</span></span><br><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">OrderServiceImpl</span> <span class="keyword">implements</span> <span class="title">OrderService</span> </span>{</span><br><span class="line"></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> OrderRepository orderRepository;</span><br><span class="line"></span><br><span class="line"> <span class="comment">//NamedParameterJdbcTemplate 更加简便的封装 和 JdbcTemplat性能差不多</span></span><br><span class="line"> <span class="meta">@Autowired</span></span><br><span class="line"> <span class="keyword">private</span> NamedParameterJdbcTemplate namedParameterJdbcTemplate;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="meta">@Transactional</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> String <span class="title">save</span><span class="params">(OrderSaveReqVo reqVo)</span> </span>{</span><br><span class="line"> SnowFlake snowFlake = <span class="keyword">new</span> SnowFlake(<span class="number">1</span>, <span class="number">1</span>);</span><br><span class="line"></span><br><span class="line"> ArrayList<Order> orders = <span class="keyword">new</span> ArrayList<Order>();</span><br><span class="line"> <span class="keyword">for</span> (<span class="keyword">int</span> i = <span class="number">0</span>; i <= <span class="number">1000</span>; i++) {</span><br><span class="line"></span><br><span class="line"> Order order = <span class="keyword">new</span> Order();</span><br><span class="line"> order.setCode(reqVo.getCode());</span><br><span class="line"> order.setName(reqVo.getName());</span><br><span class="line"> order.setRemark(reqVo.getRemark());</span><br><span class="line"> order.setPrice(reqVo.getPrice());</span><br><span class="line"> order.setUserId(reqVo.getUserId());</span><br><span class="line"> order.setId(snowFlake.nextId());</span><br><span class="line"> order.setCreateTimestamp(<span class="keyword">new</span> Timestamp(System.currentTimeMillis()));</span><br><span class="line"></span><br><span class="line"> orders.add(order);</span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> </span><br><span class="line"> SqlParameterSource[] beanSources = SqlParameterSourceUtils.createBatch(orders.toArray());</span><br><span class="line"> String sql = <span class="string">"insert into order_info (id,code,name,remark,price,user_id,create_timestamp) values (:id,:code,:name,:remark,:price,:userId,:createTimestamp)"</span>;</span><br><span class="line"> namedParameterJdbcTemplate.batchUpdate(sql, beanSources);</span><br><span class="line"> <span class="keyword">return</span> <span class="string">"成功"</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>用Postman测试时间:</p><p><img alt="enter image description here" data-src="/img/DSO/1DA166A0-A0F0-4104-9B8F-0966A28D6A08.png" class="lazyload"></p><p>不减反增!!!查询大量的资料,最终发现问题所在:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 注意 rewriteBatchedStatements=true 这个参数是开启批处理 很重要!!! 测试1W条数据没他许需要4分钟 开启只需要762ms 需要搭配JdbcTemplate使用</span></span><br><span class="line"><span class="string">spring.datasource.url=jdbc:mysql://localhost:3306/spring_data?serverTimezone=UTC&rewriteBatchedStatements=true</span></span><br></pre></td></tr></table></figure><p>加入批处理的配置:rewriteBatchedStatements=true</p><p>直接测试1W条数据:</p><p>JdbcTemplate:1061ms</p><p><img alt="enter image description here" data-src="/img/DSO/6BA83FA6-5A22-478e-BCE7-17A990B5DD41.png" class="lazyload"></p><p>JPA:2.4s</p><p><img alt="enter image description here" data-src="/img/DSO/4CB3FEE3-7081-4822-BE9A-EC2D4EA65942.png" class="lazyload"></p><p>解决!!!</p><p>也可以分析出来,大数据(和1K的数据相比)存储性能上来看,JdbcTemplate性能要大大好于JPA的,但我还是喜欢JPA,JPA真香!!</p><p>延伸:<br>我后面还做的10W,100W,1000W的数据测试:</p><table><thead><tr><th>数据量</th><th>10W</th><th>100W</th><th>1000W</th></tr></thead><tbody><tr><td>JPA</td><td>38s</td><td>437s</td><td>…</td></tr><tr><td>JdbcTemplate</td><td>3.25s</td><td>48s</td><td>…</td></tr></tbody></table><p>1000W数据时会报错:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">java.lang.OutOfMemoryError: GC overhead limit exceeded</span><br><span class="line">...</span><br></pre></td></tr></table></figure><p>这是在快OOM之前JVM的保护措施,代表垃圾回收耗费了98%的时间,但是回收的内存还不到2%,那么JVM会认为即将发生OOM,让程序提前结束。</p><p>可以看出,JdbcTemplate的存储性能确实远远好于Jpa,但是也有瓶颈;在上百万级别的数据可以考虑两种方案进行优化:<br>redis和多线程批量保存。这两种方案在后面我做好数据统计会再发出来,今天暂不赘述;</p><h4 id="问题"><a href="#问题" class="headerlink" title="问题"></a>问题</h4><p>值得多说的是我在测试数据的时候遇到的一些问题,在这里记录一下:</p><p>问题一:</p><p>启动Zuul服务请求超时解决:</p><p>开始时 zuul 的 application.properties:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">server.port=8083</span></span><br><span class="line"><span class="string">spring.application.name=zuul</span></span><br><span class="line"><span class="string">eureka.client.service-url.defaultZone=http://localhost:8761/eureka/</span></span><br><span class="line"><span class="string">zuul.routes.zuul.path=/hello/**</span></span><br><span class="line"><span class="string">zuul.routes.zuul.service-id=order</span></span><br></pre></td></tr></table></figure><p>刚开始测试阶段(没有开启加批处理),我在保存100条数据需要10s左右,就会报网关超时;</p><p><img alt="enter image description here" data-src="/img/DSO/0454F32F-F401-4ac8-8142-655EB684B9F7.png" class="lazyload"></p><p>后面改了 application.properties 配置 新增了这两个参数:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">zuul.host.connect-timeout-millis=50000</span></span><br><span class="line"><span class="string">zuul.host.socket-timeout-millis=60000</span></span><br></pre></td></tr></table></figure><p>但是不生效,测试不到2s就已经报错;</p><p>后来查阅资料才知道,原来如果 application.properties 配置指定的服务(service-id)那就会走ribbon负载均衡,修改配置文件:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="string">server.port=8083</span></span><br><span class="line"><span class="string">spring.application.name=zuul</span></span><br><span class="line"><span class="string">eureka.client.service-url.defaultZone=http://localhost:8761/eureka/</span></span><br><span class="line"><span class="string">zuul.routes.zuul.path=/hello/**</span></span><br><span class="line"><span class="string">zuul.routes.zuul.service-id=order</span></span><br><span class="line"><span class="string">zuul.host.connect-timeout-millis=50000</span></span><br><span class="line"><span class="string">zuul.host.socket-timeout-millis=60000</span></span><br><span class="line"><span class="string">ribbon.ConnectTimeout=3000</span></span><br><span class="line"><span class="string">ribbon.ReadTimeout=10000</span></span><br></pre></td></tr></table></figure><p>注意:</p><p>如果指定了 “zuul.routes.zuul.service-id”这个参数指定得服务,就会走ribbon负载均衡,只有直接指定具体的url才会走zuul网关;</p><p>问题二:</p><p>在使用JPA 执行deleteAll()时速度过慢:</p><p>设置orderService服务application.properties参数:</p><figure class="highlight yaml"><table><tr><td class="code"><pre><span class="line"><span class="comment"># 是否显示SQL语句</span></span><br><span class="line"><span class="string">spring.jpa.show-sql=true</span></span><br></pre></td></tr></table></figure><p>发现JPA执行deleteAll()时,其实是一条一条删的…..不慢才怪;</p><p>改造:</p><p>service层:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"> <span class="meta">@Override</span></span><br><span class="line"> <span class="meta">@Transactional</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title">delete</span><span class="params">()</span> </span>{</span><br><span class="line"> orderRepository.deleteOrderInfo();</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>OrderRepository:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="meta">@Modifying</span></span><br><span class="line"><span class="meta">@Query</span>(value = <span class="string">"DELETE FROM order_info"</span>, nativeQuery = <span class="keyword">true</span>)</span><br><span class="line"><span class="function"><span class="keyword">void</span> <span class="title">deleteOrderInfo</span><span class="params">()</span></span>;</span><br></pre></td></tr></table></figure><p>需要注意的是:@Modifying 和 @Transactional 这两个注解;</p><p>@Modifying:表明需要对数据进行修改操作;一般搭配@Query注解一起使用;</p><p>@Transactional :声明式事务管理,对数据库数据进行修改操作时需要提交事务,不加这个注解提交事务会异常;</p><p>持续更新<del>~</del></p>]]></content>
<categories>
<category> 数据库 </category>
</categories>
<tags>
<tag> 优化 </tag>
<tag> 数据库 </tag>
</tags>
</entry>
<entry>
<title>微架构SpringCloud-Eureka服务注册中心</title>
<link href="/2019/11/15/%E5%88%86%E5%B8%83%E5%BC%8F/Eureka/"/>
<url>/2019/11/15/%E5%88%86%E5%B8%83%E5%BC%8F/Eureka/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="微架构SpringCloud-Eureka服务注册中心"><a href="#微架构SpringCloud-Eureka服务注册中心" class="headerlink" title="微架构SpringCloud-Eureka服务注册中心"></a>微架构SpringCloud-Eureka服务注册中心</h2><p> Erueka是一个基于REST,服务注册与发现的组件。<br> 它主要包括两个组件:</p><blockquote><p>· Eureka Client: 一个Java客户端,用于简化与 Eureka Server 的交互(通常就是微服务中的客户端和服务端)<br>· Eureka Server:提供服务注册和发现的能力(通常就是微服务中的注册中心)</p></blockquote><h5 id="Eureka-的作用"><a href="#Eureka-的作用" class="headerlink" title="Eureka 的作用"></a>Eureka 的作用</h5><blockquote><p>开发过程中,各个服务之间的调用来实现业务逻辑不同走向,而如果将所有的服务都放进一个工程里就会出现项目工程的大量冗余及各种资源的叠加。就像是将所有年纪的学生放进一个教室里,造成项目极难管理,甚至阻碍了开发。那么我们就可以将不同的服务拆分开来,专注与某个服务逐一管理,微服务由此而生。各个服务之间相互的通信完成不同的业务逻辑走向,但是随着服务的不断增长,那服务之间的通信也就会出现相对松散,链路也就相对复杂。举个例子:我们去机场取票,我们肯定是去对应的航空公司取票。但是如果机场很大,又有很多家航空公司,你咋办呢?只能一家一家去找?我想应该会去找服务站的人咨询吧。微服务也是一样,而Eureka就充当这服务站的角色;他会告诉你,你所需要的服务(对应的航空公司)在哪里,是不是就做到了服务的链路简便(节约时间)。</p></blockquote><p><img alt="enter image description here" data-src="/img/Eureka/A593C580-086A-4da4-B79E-01CC6CAC344A.png" class="lazyload"></p><h5 id="搭建一个Eureka服务中心"><a href="#搭建一个Eureka服务中心" class="headerlink" title="搭建一个Eureka服务中心"></a>搭建一个Eureka服务中心</h5><p>参考:<a href="https://www.jianshu.com/p/c89e2479595d" target="_blank" rel="noopener">https://www.jianshu.com/p/c89e2479595d</a></p><p>注意:</p><p>我的项目结构如下:</p><p><img alt="enter image description here" data-src="/img/Eureka/86BF389E-25BC-4bbc-9845-AF083A6D9C3E.png" class="lazyload"></p><p>EurekaServer启动成功,再启动两个服务的时候会报错:</p><p><img alt="enter image description here" data-src="/img/Eureka/BCBDA4F0-A84F-45d7-AD6B-8D12B0B4B177.png" class="lazyload"></p><p>报错提示:端口被占用,需要指定两个服务的端口<br><img alt="enter image description here" data-src="/img/Eureka/879B2DA4-BC24-4d42-A3D2-8478FB220409.png" class="lazyload"><br><img alt="enter image description here" data-src="/img/Eureka/0DA6EB6E-5DD2-4893-A560-0234D1471C12.png" class="lazyload"></p><h5 id="Eureka集群"><a href="#Eureka集群" class="headerlink" title="Eureka集群"></a>Eureka集群</h5><blockquote><p>集群概念:集群通信系统是一种用于集团调度指挥通信的移动通信系统,主要应用在专业移动通信领域。该系统具有的可用信道可为系统的全体用户共用,具有自动选择信道功能,它是共享资源、分担费用、共用信道设备及服务的多用途、高效能的无线调度通信系统。<br>人话:小时候老师罚抄10遍课文,一个人要抄10遍,一遍需要10分钟,抄完需要100分钟。但是,这时你的小伙伴有10个,一人抄一遍,抄完只需要十分钟。</p></blockquote><p>Eureka集群也一样,回到前面的例子,这时你到了服务站(EurekaServer),正好赶上高峰期,你前面排了一群人都在咨询,一群人都在催,服务站慢慢的也疲于应付最后导致崩溃,对应到计算机就是服务器宕机,哦吼,全完蛋,整个应用系统全部瘫痪;<br><img alt="enter image description here" data-src="/img/Eureka/D9D930CF-6E83-44f0-AF1F-1753D2263728.png" class="lazyload"></p><p>方案:<br>咋办呢?一直等下去?插队可耻不可取。这时机场做出了对应措施,增加服务站。对!没错,对应的就是增加EurekaSever,将客户请求分流;</p><p><img alt="enter image description here" data-src="/img/Eureka/4B4E64B6-1E81-48d6-8C14-49E0A00931D5.png" class="lazyload"></p><h5 id="搭建一个Eureka集群服务注册中心"><a href="#搭建一个Eureka集群服务注册中心" class="headerlink" title="搭建一个Eureka集群服务注册中心"></a>搭建一个Eureka集群服务注册中心</h5><p>参考:<a href="https://www.jianshu.com/p/c7b13e06bd46" target="_blank" rel="noopener">https://www.jianshu.com/p/c7b13e06bd46</a></p><h5 id="Eureka应用集群"><a href="#Eureka应用集群" class="headerlink" title="Eureka应用集群"></a>Eureka应用集群</h5><p>再再回到前面的例子,这时候你终于到了对应的航空公司准备换登机牌,嗯,赶巧了,换登机牌也在排队…..对!没错,一样的道理,开设新的换登机牌服务站;</p><p><img alt="enter image description here" data-src="/img/Eureka/0090A2D3-1100-42ce-9059-6C972EF391EB.png" class="lazyload"></p><h5 id="搭建一个Eureka集群服务应用中心"><a href="#搭建一个Eureka集群服务应用中心" class="headerlink" title="搭建一个Eureka集群服务应用中心"></a>搭建一个Eureka集群服务应用中心</h5><p>参考:<a href="https://www.jianshu.com/p/c7b13e06bd46" target="_blank" rel="noopener">https://www.jianshu.com/p/c7b13e06bd46</a></p><h5 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h5><p>这样基本上就实现了Eureka集群,达到集群的高可用的特点;Eureka还给我们提供了很多的高可用的机制,在后面使用的过程中也会边学边更新进来。</p><p>知识点:<br>Eureka,它实际上起到的是一个注册服务!它就像一个云端通讯录!应用启动,注册到EurekaSever上,同时也获取了其他云端成云的联系信息【并缓存到本应用信息】,所以若“基站”服务突然不存在了,并不妨碍宕机之前以注册到“基站”应用间的通信!但若一个新的应用也试着注册到Eureka,首先,它无法注册,因为“基站”服务已经不存在了,其二,那么新注册的应用也无法与之前的应用进行通信,因为它们之间没有在一个云端里交流!若想更新“通讯方式”,需重启Eureka服务以自动捕获Eureka客户端刷新“通讯录”。</p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> 微服务 </tag>
</tags>
</entry>
<entry>
<title>记录我心中Top10的电影</title>
<link href="/2019/11/10/%E7%94%B5%E5%BD%B1/%E6%88%91%E5%BF%83%E4%B8%ADTop10%E7%9A%84%E7%94%B5%E5%BD%B1/"/>
<url>/2019/11/10/%E7%94%B5%E5%BD%B1/%E6%88%91%E5%BF%83%E4%B8%ADTop10%E7%9A%84%E7%94%B5%E5%BD%B1/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="Top10:久石让武道馆音乐会"><a href="#Top10:久石让武道馆音乐会" class="headerlink" title="Top10:久石让武道馆音乐会"></a>Top10:久石让武道馆音乐会</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/9SXZ3PYrM5sF4el.jpg" class="lazyload"></p><blockquote><p>三年前我看了一个片段,好像是幽灵公主,然后就一直存在手机里了。我一直都很喜欢宫崎骏的动漫,后面也看过几部让我印象深刻的电影,神奇的是有好几部都是久石让配的乐。也让我明白了音乐就是一部电影的灵魂,而久石让的音乐给我的感觉就是”纯粹”;</p></blockquote><h2 id="Top9-第六感生死缘"><a href="#Top9-第六感生死缘" class="headerlink" title="Top9: 第六感生死缘"></a>Top9: 第六感生死缘</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/3Zg7YPKb498wFu6.jpg" class="lazyload"></p><blockquote><p>初中在红山电影频道看了后半段,当时啥也不懂,只觉得朦朦胧胧的女主很好看。最后死神和比尔走下了桥,布莱克从桥的那边走了回来只觉得心里有种失落感,后来想从头看忘了名字,多亏皮特演的电影多。从头看了一遍还专门在快播写了第一次影评,现在还记得我写的有句话:”如果说这部电影的剧情是躯体,那配乐就是注入躯体的灵魂”。zuo~</p></blockquote><h2 id="Top8-互联网之子"><a href="#Top8-互联网之子" class="headerlink" title="Top8:互联网之子"></a>Top8:互联网之子</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/blHXJxwQMnUve2A.png" class="lazyload"></p><blockquote><p>讲述了亚伦·斯沃茨开挂的一生,有趣的是现在我写博客用的Markdown就是他和另一人共同设计编写的。我们都是站在巨人的肩膀上发现世界的;</p></blockquote><h2 id="Top7-英雄"><a href="#Top7-英雄" class="headerlink" title="Top7:英雄"></a>Top7:英雄</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/QJFPuvaqxr7fU89.png" class="lazyload"></p><blockquote><p>老谋子的二板斧,”活着”没看过,”英雄”算是侠义片中的精品了,我觉得是第一了。”东邪西毒”是爱情片。为此专门去了趟九寨沟,很美,但还是和电影差点。所以这部电影,不光在剧情上是巨佳,在艺术,意境,配乐上都是佳品!小我。天下。</p></blockquote><h2 id="Top6-海洋"><a href="#Top6-海洋" class="headerlink" title="Top6:海洋"></a>Top6:海洋</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/A1ydtVExDmSazh5.jpg" class="lazyload"></p><blockquote><p>纪录片的魅力在于可以用第一人称的视角去感受不一样的人生或着他生。蓝色的星球,蓝色代表着海洋,海洋拥抱着陆地。在自然的面前人类承认自我的渺小才是对生命的尊重吧。配乐满分,部分配乐还作为了《巫师3》CG动画的插曲;</p></blockquote><h2 id="Top5-不能说得秘密"><a href="#Top5-不能说得秘密" class="headerlink" title="Top5:不能说得秘密"></a>Top5:不能说得秘密</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/11/FPvlmCkJoOsEB1w.png" class="lazyload"></p><blockquote><p>周董的处女座,我的启蒙片,哈哈哈哈哈哈哈,小学看完的我有点蒙,一度非常的迷恋这部电影,天天下课借我同桌的手机听《不能说的秘密》,后来顺利的将这位同桌变成了我的初恋。现在能保持每天吃一个苹果全托了”An apple a day keeps the doctor away.”;好的电影对于我就是能回忆起那时的自己那时的他们,让我感叹岁月蹉跎的同时还能念起你催我换手机的样子。</p></blockquote><h2 id="Top4:完美陌生人"><a href="#Top4:完美陌生人" class="headerlink" title="Top4:完美陌生人"></a>Top4:完美陌生人</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/11/G45aUtrXfpA7Pui.png" class="lazyload"></p><blockquote><p>唯一一部不靠配乐进来的,可以想像内容有多硬核。想了半天怎么表述,总之硬核。</p></blockquote><h2 id="Top3-哈利波特-死亡圣器(上下)"><a href="#Top3-哈利波特-死亡圣器(上下)" class="headerlink" title="Top3:哈利波特-死亡圣器(上下)"></a>Top3:哈利波特-死亡圣器(上下)</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/11/3ecIzQCNuTWOYMR.png" class="lazyload"></p><blockquote><p>小时候的我觉得哈利波特是个魔幻冒险片,慢慢长大了,哈利也从白净的孩子长成了络腮胡,我也是,但比他帅。随着《火焰杯》虫尾巴对塞德里克使用了”阿瓦达索命”开始,《哈利波特》正式告别了从前,成为了一部恐怖的压抑的残酷的成长电影。死亡圣器上是初中我月考刚考完一个人去看的,在南门电影院,因为最后多比死了,加上考砸了,电影结束后我第一次感受到了压抑;死亡圣器下好像是和李智皓和柴世锦一起看的,电影结束后我们三个坐在椅子上愣着没走。《哈利波特》的终结,代表着我们也正是告别了童年,以后的我们也将随人海渐行渐远最后消失。我是幸运的,老李还在。感谢他对本文章100块的打赏,懂咩?</p></blockquote><h2 id="Top2-恶人"><a href="#Top2-恶人" class="headerlink" title="Top2:恶人"></a>Top2:恶人</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/11/B9izpYQdHljP2RO.jpg" class="lazyload"></p><blockquote><p>久石让配乐,艺术源于生活,越是贴近生活越能给你会心一击。这部电影豆瓣评分并不高,但是确实给了我很久的负能量。在背阴处呆久了的人看到初升的朝阳都会流泪吧。</p></blockquote><h2 id="Top1-星际穿越"><a href="#Top1-星际穿越" class="headerlink" title="Top1:星际穿越"></a>Top1:星际穿越</h2><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/11/WLE5XnYZDKmdNtI.jpg" class="lazyload"></p><blockquote><p>-. . .– - — -. … - …. .. .-. -.. .-.. .- .– </p></blockquote>]]></content>
<categories>
<category> 电影 </category>
</categories>
<tags>
<tag> 电影 </tag>
</tags>
</entry>
<entry>
<title>Java类加载机制<一> — 类加载器</title>
<link href="/2019/11/10/Java/Java%E7%B1%BB%E5%8A%A0%E8%BD%BD%E6%9C%BA%E5%88%B6-%E7%B1%BB%E5%8A%A0%E8%BD%BD%E5%99%A8/"/>
<url>/2019/11/10/Java/Java%E7%B1%BB%E5%8A%A0%E8%BD%BD%E6%9C%BA%E5%88%B6-%E7%B1%BB%E5%8A%A0%E8%BD%BD%E5%99%A8/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="Java类加载机制-lt-一-gt-—-类加载器"><a href="#Java类加载机制-lt-一-gt-—-类加载器" class="headerlink" title="Java类加载机制<一> — 类加载器"></a>Java类加载机制<一> — 类加载器</h2><blockquote><p>虚拟机把描述类的数据从Class文件加载到内存,并对数据进行校验、转换解析和初始化,最终形成可以被虚拟机直接使用的Java类型,这就是虚拟机类加载机制。在Java语言中,类的加载、连接和初始化过程都是在程序运行期间完成的,这是java作为动态语言的基础。另外值得注意的是上面提到的Class文件,并不一定值得是磁盘上的.class文件,而只需要是任何符合字节码规范的一串二进制字节流就可以了。</p></blockquote><h5 id="类加载过程:"><a href="#类加载过程:" class="headerlink" title="类加载过程:"></a>类加载过程:</h5><blockquote><p>类从被加载到虚拟机内存开始,到卸载出内存为止,它的整个生命周期包括:加载、验证、准备、解析、初始化、使用和卸载7个阶段。其中验证、准备、解析三个阶段称之为连接。<br><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/ZO6sBeQXhGEcPFx.jpg" class="lazyload"></p></blockquote><h5 id="类加载器:"><a href="#类加载器:" class="headerlink" title="类加载器:"></a>类加载器:</h5><blockquote><p>类加载由类加载器完成。</p><p>类加载器负责将可能是网络上,或者磁盘中的class文件加载到内存中,并为其生成对应的java,lang,class对象。一旦一个类已经被jvm加载完成那么这个类就不会再被加载了,那么怎么才能确认一个类只能被加载一次?在java中,一个类用其全限定类名(包名 + 类名)作为其唯一标识,但在JVM中采取的是“全限定类名”+ “类加载器”作为唯一标识。也就是说,一个类如果是不同的类加载器加载,那么生成的class文件是不同的。</p><p>举个栗子:现在在loader包下面有一个Test的类,被类加载器ClassLoader的实例Kl负责加载,则该Test类在对应的class对象JVM中表示为(Test.loader.Kl),那么如果该类被类加载器ClassLoader的实例Kl2加载时,那么Test类对应的class对象在JVM中表示为(Test.loader.Kl2)。这意味着两个同名类的类加载器(Test.loader.Kl)和(Test.loader.Kl2)是不同的,他们互不兼容。</p><p>JVM预定义有三种类加载器,当JVM开始启动时,Java开始使用以下三种类加载器:</p><blockquote><p>1.启动类加载器(根类加载器):BootstrapClassLoader<br>是嵌在JVM内核中的加载器,该加载器是用C++语言写的,主要负载加载JAVA_HOME/lib下的类库,启动类加载器无法被应用程序直接使用。</p></blockquote></blockquote><blockquote><blockquote><p>2.扩展类加载器:Extension ClassLoader<br>Java编写,且他的父类加载器是Bootstrap, 是由sun.misc.Launcher$ExtClassLoader实现的,主要加载JAVA_HOME/lib/ext目录中的类库。开发者可以使用扩展类加载器。</p></blockquote></blockquote><blockquote><blockquote><p>3.系统类加载器:App ClassLoader<br>系统类加载器,也称为应用程序类加载器,它负责在JVM启动时加载来自Java命令的-classpath选项、java.class.path系统属性,或者CLASSPATH换将变量所指定的JAR包和类路径。程序可以通过ClassLoader的静态方法getSystemClassLoader()来获取系统类加载器。如果没有特别指定,则用户自定义的类加载器都以此类加载器作为父加载器。它的父加载器为Ext ClassLoader。</p></blockquote></blockquote><p>上述三者的层级关系如下图:<br><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/wobQOWUN8L7IfGs.png" class="lazyload"></p><p>需要注意的是:类加载的体系并不是“继承”体系,而是“委托”体系。 大多数类加载器首先会到自己的parent中查找类或者资源,如果找不到才会到自己本地查找。类加载器的委托行为动机是为了避免相同的类被加载多次。</p><p>下面我们通过程序来验证:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line">System.out.println(ClassLoader.getSystemClassLoader().getParent());</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出的结果是:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">sun.misc.Launcher$ExtClassLoader@<span class="number">42e816</span></span><br></pre></td></tr></table></figure><p>对比三者的层级关系图来看,App ClassLoader 的父类加载器确实是Ext ClassLoader,那么我们如果再往上走一层,猜想没错的话,输出的结果应该是Bootstrap ClassLoader</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">main</span><span class="params">(String[] args)</span></span></span><br><span class="line"><span class="function"></span>{</span><br><span class="line">System.out.println(ClassLoader.getSystemClassLoader().getParent().getParent());</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>输出的结果是:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">null</span></span><br></pre></td></tr></table></figure><p>注意:这里不是说 Ext ClassLoader 没有父类加载器,而是因为Bootstrap ClassLoader是C++写的;</p><p>下面我们从源码中看看他们之间的继承关系:</p><p><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/rFWlAoh2Pw3K8eZ.png" class="lazyload"></p><p>结论:除了启动类加载器Bootstrap ClassLoader,其他类加载器都是ClassLoader的子类;</p><p>再来看看 AppClassLoader的源码:<br>可以看出,系统加载器:AppClassLoader只能加载到指定的“java.class.path”路径下的Class文件。</p><h5 id="双亲委派机制:"><a href="#双亲委派机制:" class="headerlink" title="双亲委派机制:"></a>双亲委派机制:</h5><blockquote><p>如果一个类加载器收到了一个类加载请求,它不会自己去尝试加载这个类,而是把这个请求转交给父类加载器去完成。每一个层次的类加载器都是如此。因此所有的类加载请求都应该传递到最顶层的启动类加载器中,只有到父类加载器反馈自己无法完成这个加载请求(在它的搜索范围没有找到这个类)时,子类加载器才会尝试自己去加载。委派的好处就是避免有些类被重复加载。</p></blockquote><p>双亲委派的实现比较简单,我们来看下源码:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">synchronized</span> Class<?> loadClass(String name, <span class="keyword">boolean</span> resolve) </span><br><span class="line"><span class="keyword">throws</span> ClassNotFoundException </span><br><span class="line">{ </span><br><span class="line"><span class="comment">// First, check if the class has already been loaded </span></span><br><span class="line">Class c = findLoadedClass(name); </span><br><span class="line"> <span class="comment">//该类还没有被加载,那么则 </span></span><br><span class="line"><span class="keyword">if</span> (c == <span class="keyword">null</span>) { </span><br><span class="line"><span class="keyword">try</span> { </span><br><span class="line"> <span class="comment">//如其父类不为空,那么则使用其父类进行加载(父类委托机制) </span></span><br><span class="line"> <span class="keyword">if</span> (parent != <span class="keyword">null</span>) { </span><br><span class="line"> c = parent.loadClass(name, <span class="keyword">false</span>); </span><br><span class="line"> } </span><br><span class="line"> <span class="comment">//如果其父类为空则使用BootstrapClass进行加载 </span></span><br><span class="line"> <span class="keyword">else</span> { </span><br><span class="line"> c = findBootstrapClass0(name); </span><br><span class="line"> } </span><br><span class="line"> } <span class="keyword">catch</span> (ClassNotFoundException e) { </span><br><span class="line"> <span class="comment">// If still not found, then invoke findClass in order // to find the class. </span></span><br><span class="line"> <span class="comment">//如果此类的父类加载器无法加载该类,那么由此类加载器的findClass进行类字节码的加载 </span></span><br><span class="line"> c = findClass(name); </span><br><span class="line"> } </span><br><span class="line">} </span><br><span class="line"> <span class="comment">//是否需要解析类,如果resolve=true时,则保证已经装载,而且已经连接了。resolve=falses时,则仅仅是去装载这个类,不关心是否连接了,所以此时可能被连接了,也可能没有被连接 </span></span><br><span class="line"><span class="keyword">if</span> (resolve) { </span><br><span class="line"> resolveClass(c); </span><br><span class="line">} </span><br><span class="line"><span class="keyword">return</span> c; </span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>先检查是否已经被加载过,若没有加载则调用父类加载器的loadClass方法,若父类加载器不存在,则使用启动类加载器。如果父类加载器加载失败,则抛出异常之后看,再调用自己的findClass方法进行加载。</p><p>除了上面介绍的的loadClass方法下面在介绍几个比较重要的方法:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">protected</span> <span class="keyword">final</span> Class<?> defineClass </span><br><span class="line"><span class="keyword">protected</span> Class<?> findClass(String name) </span><br><span class="line"><span class="function"><span class="keyword">protected</span> <span class="keyword">final</span> <span class="keyword">void</span> <span class="title">resolveClass</span><span class="params">(Class<?> c)</span></span></span><br></pre></td></tr></table></figure><blockquote><p>上面的loadClass方法的作用只是启动类的加载,指定类应该由谁进行加载。而具体的怎么加载需要用到defineClass方法:其能够将byte字节流解析成JVM能够识别的Class对象。</p><p>findClass:负责取得需要加载的类的字节码,然后可以调用definedClass方法生成类的Class对象。有了这样两个方法我们不仅仅可以通过class文件实例化对象,还可以通过网络接受字节码实例化对象。一般我们配合使用defineClass和findClass方法,直接覆盖ClassLoader父类的findClass方法来实现类的加载规则,findClass取得需要加载的类的字节码,defineClass根据字节码创建类对象;</p></blockquote>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
</tags>
</entry>
<entry>
<title>Java数据机构-链表特性</title>
<link href="/2019/11/10/Java/Java%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-%E9%9B%86%E5%90%88%E7%9B%B8%E5%85%B3/"/>
<url>/2019/11/10/Java/Java%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84-%E9%9B%86%E5%90%88%E7%9B%B8%E5%85%B3/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="Java数据机构-链表特性"><a href="#Java数据机构-链表特性" class="headerlink" title="Java数据机构-链表特性"></a>Java数据机构-链表特性</h2><h5 id="链表的特点:"><a href="#链表的特点:" class="headerlink" title="链表的特点:"></a>链表的特点:</h5><blockquote><p>链表是以分散的形式存储在内存中的,在查询元素时只能通过上个元素中存储的下个元素的信息进行查询。也就是说,链表不支持随机访问,要想查询某个元素,只能从头节点开始遍历。时间复杂度相较于数组的随机访问会增大;</p></blockquote><h5 id="时间复杂度:"><a href="#时间复杂度:" class="headerlink" title="时间复杂度:"></a>时间复杂度:</h5><blockquote><p>在描述算法复杂度时,经常用到o(1), o(n), o(logn), o(nlogn)来表示对应算法的时间复杂度。这里进行归纳一下它们代表的含义:这是算法的时空复杂度的表示。不仅仅用于表示时间复杂度,也用于表示空间复杂度。</p></blockquote><blockquote><p>O后面的括号中有一个函数,指明某个算法的耗时/耗空间与数据增长量之间的关系。其中的n代表输入数据的量。</p><p>比如时间复杂度为O(n),就代表数据量增大几倍,耗时也增大几倍。比如常见的遍历算法。</p><p>再比如时间复杂度O(n^2),就代表数据量增大n倍时,耗时增大n的平方倍,这是比线性更高的时间复杂度。比如冒泡排序,就是典型的O(n^2)的算法,对n个数排序,需要扫描n×n次。</p><p>再比如O(logn),当数据增大n倍时,耗时增大logn倍(这里的log是以2为底的,比如,当数据增大256倍时,耗时只增大8倍,是比线性还要低的时间复杂度)。二分查找就是O(logn)的算法,每找一次排除一半的可能,256个数据中查找只要找8次就可以找到目标。</p><p>O(nlogn)同理,就是n乘以logn,当数据增大256倍时,耗时增大256*8=2048倍。这个复杂度高于线性低于平方。归并排序就是O(nlogn)的时间复杂度。</p><p>O(1)就是最低的时空复杂度了,也就是耗时/耗空间与输入数据大小无关,无论输入数据增大多少倍,耗时/耗空间都不变。 哈希算法就是典型的O(1)时间复杂度,无论数据规模多大,都可以在一次计算后找到目标(不考虑冲突的话)</p></blockquote><h5 id="链表分为单链表和双向列表"><a href="#链表分为单链表和双向列表" class="headerlink" title="链表分为单链表和双向列表"></a>链表分为单链表和双向列表</h5><p>共同点:</p><blockquote><p>1.无法进行随机访问<br>2.他们都能够在O(1)时间内在给定的节点或在列表开头添加一个新节点<br>3.他们都能够在O(1)时间内删除列表的第一个节点</p><p>但在删除一个给定的节点(包括最后一个节点)时略有不同单向列表在删除一个给定节点时只能从第一个节点依次遍历进行查找删除,在时间复杂度上是O(n),而双向列表因为记录了前驱节点的信息所以我们可以在O(1)的时间内删除给定的节点</p></blockquote><p>优点:</p><blockquote><p>1.因为链表在内存中储存的形式为不连续的,所以在内存空间利用上会比较高,并且在删除和插入操作上时间复杂度O(1),相比与数组会快<br>2.链表支持动态的扩容</p></blockquote><p>缺点:</p><blockquote><p>1.因为链表的节点会存储下个节点的信息(双向链表会存储上个节点的信息)所以一个节点的内存占用会比数组大<br>2.链表的查询这只能从头节点依次向后遍历,不支持随机访问,所有查询的效率会很低</p></blockquote><p>总结:</p><blockquote><p>1.如果对数据要经常进行删除或插入的操作建议使用链表<br>2.如果对数据只进行查询并且数据的大小已经确定建议使用数组</p></blockquote><p>各个数据结构的常见操作及排序算法的时间复杂度汇总:<br><img alt="enter image description here" data-src="https://i.loli.net/2019/11/10/HutP7dzG3b9xpae.png" class="lazyload"></p><p>图片源自:<a href="https://blog.csdn.net/weixin_44387066/article/details/87694330" target="_blank" rel="noopener">https://blog.csdn.net/weixin_44387066/article/details/87694330</a></p>]]></content>
<categories>
<category> Java </category>
</categories>
<tags>
<tag> Java </tag>
<tag> 数据结构 </tag>
</tags>
</entry>
<entry>
<title>分布式锁</title>
<link href="/2019/11/10/%E5%88%86%E5%B8%83%E5%BC%8F/%E5%88%86%E5%B8%83%E5%BC%8F%E9%94%81/"/>
<url>/2019/11/10/%E5%88%86%E5%B8%83%E5%BC%8F/%E5%88%86%E5%B8%83%E5%BC%8F%E9%94%81/</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="Java分布式锁"><a href="#Java分布式锁" class="headerlink" title="Java分布式锁"></a>Java分布式锁</h2><h5 id="为什么要使用分布式锁:"><a href="#为什么要使用分布式锁:" class="headerlink" title="为什么要使用分布式锁:"></a>为什么要使用分布式锁:</h5><blockquote><p>为了保证一个方法或属性在高并发的情况下同一时间只能被同一个线程执行,在传统单机部署的情况下,可以使用Java并发处理相关的API(如ReentrantLock或Synchronized)进行互斥控制。但是,随之业务发展的需要,原单机部署的系统演化成分布式集群系统后,由于分布式系统多线程、多进程并且分布在不同的机器上,这将原来的单机部署情况下的并发控制锁策略失效,单纯的Java API并不能提供分布式锁的能力。</p><p>为了解决这个问题,就需要一种跨JVM的互斥机制来控制共享资源的访问,这就是分布式锁要解决的问题!</p></blockquote><h5 id="分布式锁应该具备哪些条件"><a href="#分布式锁应该具备哪些条件" class="headerlink" title="分布式锁应该具备哪些条件"></a>分布式锁应该具备哪些条件</h5><blockquote><ul><li>在分布式系统环境下,一个方法在同一时间只能被一个机器的一个线程执行;</li><li>高可用、高性能的获取锁与释放锁;</li><li>具备可重入特性;</li><li>具备锁失效机制、防止死锁;</li><li>具备非阻塞锁特性,即没有获取到锁直接返回获取锁失败;</li></ul></blockquote><h5 id="分布式锁的实现方式"><a href="#分布式锁的实现方式" class="headerlink" title="分布式锁的实现方式"></a>分布式锁的实现方式</h5><blockquote><p>目前几乎所有大型网站及应用都是分布式部署,分布式场景中的数据一致性问题一直是一个比较重要的话题,分布式的CAP 理论告诉我们任何一个分布式系统都无法同时满足一致性(Consistency)、可用性(Availability)和分区容错性(Partition tolerance),最多只能同时满足两项。</p></blockquote><blockquote><p>一般情况下,都需要牺牲强一致性来换取系统的高可用性,系统往往只需要保证最终一致性,只要这个最终时间是在用户可以接受的范围内即可。</p></blockquote><blockquote><p>在很多时候,为了保证数据的最终一致性,需要很多的技术方案来支持,比如分布式事务、分布式锁等。有的时候,我们需要保证一信方法在同一时间内只能被同一个线程执行。</p></blockquote><blockquote><p>而分布式锁的具体实现方案有如下三种:</p><ul><li>基于数据库实现;</li><li>基于缓存(Redis等)实现;</li><li>基于Zookeeper实现;</li></ul></blockquote><blockquote><p>以上尽管有三种方案,但是我们需要根据不同的业务进行选型。</p></blockquote><h5 id="基于数据库的实现方式"><a href="#基于数据库的实现方式" class="headerlink" title="基于数据库的实现方式"></a>基于数据库的实现方式</h5><blockquote><p>基于数据库的实现方式的思想核心为:<br>在数据库中创建一个表,表中包含方法名等字段,并在方法名字段上创建唯一索引,想要执行某个方法,就使用这个方法名向表中插入数据,成功插入则获取锁,执行完成后删除对应的行数据释放锁。</p></blockquote><p> 一:创建一张表:</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">DROP</span> <span class="keyword">TABLE</span> <span class="keyword">IF</span> <span class="keyword">EXISTS</span> <span class="string">`method_lock`</span>;</span><br><span class="line"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> <span class="string">`method_lock`</span> (</span><br><span class="line"> <span class="string">`id`</span> <span class="built_in">INT</span>(<span class="number">11</span>) <span class="keyword">UNSIGNED</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> AUTO_INCREMENT</span><br><span class="line"> <span class="keyword">COMMENT</span> <span class="string">'主键'</span>,</span><br><span class="line"> <span class="string">`method_name`</span> <span class="built_in">VARCHAR</span>(<span class="number">64</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span></span><br><span class="line"> <span class="keyword">COMMENT</span> <span class="string">'锁定的方法名'</span>,</span><br><span class="line"> <span class="string">`desc`</span> <span class="built_in">VARCHAR</span>(<span class="number">255</span>) <span class="keyword">NOT</span> <span class="literal">NULL</span></span><br><span class="line"> <span class="keyword">COMMENT</span> <span class="string">'备注信息'</span>,</span><br><span class="line"> <span class="string">`update_time`</span> <span class="built_in">TIMESTAMP</span> <span class="keyword">NOT</span> <span class="literal">NULL</span> <span class="keyword">DEFAULT</span> <span class="keyword">CURRENT_TIMESTAMP</span> <span class="keyword">ON</span> <span class="keyword">UPDATE</span> <span class="keyword">CURRENT_TIMESTAMP</span>,</span><br><span class="line"> PRIMARY <span class="keyword">KEY</span> (<span class="string">`id`</span>),</span><br><span class="line"> <span class="keyword">UNIQUE</span> <span class="keyword">KEY</span> <span class="string">`uidx_method_name`</span> (<span class="string">`method_name`</span>) <span class="keyword">USING</span> BTREE</span><br><span class="line">)</span><br><span class="line"> <span class="keyword">ENGINE</span> = <span class="keyword">InnoDB</span></span><br><span class="line"> AUTO_INCREMENT = <span class="number">3</span></span><br><span class="line"> <span class="keyword">DEFAULT</span> <span class="keyword">CHARSET</span> = utf8</span><br><span class="line"> <span class="keyword">COMMENT</span> = <span class="string">'锁定中的方法'</span>;</span><br></pre></td></tr></table></figure><p>二:想要执行某个方法,就使用这个方法名向表中插入数据:</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> method_lock (method_name, <span class="keyword">desc</span>) <span class="keyword">VALUES</span> (<span class="string">'methodName'</span>, <span class="string">'测试的methodName'</span>);</span><br></pre></td></tr></table></figure><blockquote><p>由于我们对method_name做了唯一性约束,如果有多个请求同时提交插入操作时,数据库能确保只有一个操作可以成功,那么我们就可以认为操作成功的那个线程获得了该方法的锁,可以执行方法体中的内容。</p></blockquote><p>三、执行完成后,删除对应的行数据释放锁</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line"><span class="keyword">delete</span> <span class="keyword">from</span> method_lock <span class="keyword">where</span> method_name =<span class="string">'methodName'</span>;</span><br></pre></td></tr></table></figure><blockquote><p>这里只是基于数据库实现的一种方法(比较粗糙的一种)。</p></blockquote><p>但是对于分布式锁应该具备的条件来说,还有一些问题需要解决及优化:</p><blockquote><ul><li>因为是基于数据库实现的,数据库的可用性和性能将直接影响分布式锁的可用性及性能。所以,数据库需要双机部署、数据同步、主备切换;</li><li>它不具备可重入的特性,因为同一个线程在释放锁之前,行数据一直存在,无法再次成功插入数据。所以,需要在表中新增一列,用于记录当前获取到锁的机器和线程信息,在再次获取锁的时候,先查询表中机器和线程信息是否和当前机器线程相同,若相同则直接获取锁。</li><li>没有锁失效机制,因为有可能出现成功插入数据后,服务器宕机了,对应的数据没有被删除,当服务恢复后一直获取不到锁,所以,需要在表中新增一列,用于记录失效时间,并且需要有定时任务清除这些失效的数据;</li><li>不具备阻塞锁特性,获取不到锁直接返回失败,所以需要优化获取逻辑,循环多次去获取;</li><li>依赖数据库需要一定的资源开销,性能问题需要考虑;</li></ul></blockquote><p>数据库实现方案二:</p><blockquote><p>现在基于Mysql引擎InnoDB来实现:</p></blockquote><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">lock</span><span class="params">()</span></span>{</span><br><span class="line"> connection.setAutoCommit(<span class="keyword">false</span>)</span><br><span class="line"> <span class="keyword">while</span>(<span class="keyword">true</span>){</span><br><span class="line"> <span class="keyword">try</span>{</span><br><span class="line"> result = select * from methodLock where method_name=xxx <span class="keyword">for</span> update;</span><br><span class="line"> <span class="keyword">if</span>(result==<span class="keyword">null</span>){</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line"> }</span><br><span class="line"> }<span class="keyword">catch</span>(Exception e){</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> }</span><br><span class="line"> sleep(<span class="number">1000</span>);</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">}</span><br></pre></td></tr></table></figure><blockquote><p>在查询语句后面加“for update”这时数据库会自动给表加排他锁(注意:InnoDB再给表加排他锁时,只有通过索引检索时才会走行级锁,)</p></blockquote><h5 id="基于缓存(Redis)的实现方式"><a href="#基于缓存(Redis)的实现方式" class="headerlink" title="基于缓存(Redis)的实现方式"></a>基于缓存(Redis)的实现方式</h5><p>使用Redis实现分布式锁的理由:</p><blockquote><p>1.Redis具有很高的性能;<br>2.Redis的命令对此支持较好,实现起来很方便;</p></blockquote><p>Redis命令介绍:</p><p>SETNX:</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">// 当且仅当key不存在时,<span class="keyword">set</span>一个<span class="keyword">key</span>为val的字符串,返回<span class="number">1</span>;</span><br><span class="line">// 若<span class="keyword">key</span>存在,则什么都不做,返回<span class="number">0</span>。</span><br><span class="line">SETNX <span class="keyword">key</span> val;</span><br></pre></td></tr></table></figure><p>expire:</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">// 为key设置一个超时时间,单位为second,超过这个时间锁会自动释放,避免死锁。</span><br><span class="line">expire key timeout;</span><br></pre></td></tr></table></figure><p>delete:</p><figure class="highlight sql"><table><tr><td class="code"><pre><span class="line">// 删除key</span><br><span class="line"><span class="keyword">delete</span> <span class="keyword">key</span>;</span><br></pre></td></tr></table></figure><p>我们通过Redis实现分布式锁时,主要通过上面的这三个命令。</p><p>通过Redis实现分布式的核心思想为:</p><blockquote><p>1.获取锁的时候,使用setnx加锁,并使用expire命令为锁添加一个超时时间,超过该时间自动释放锁,锁的value值为一个随 机生成的UUID,通过这个value值,在释放锁的时候进行判断。<br>2.获取锁的时候还设置一个获取的超时时间,若超过这个时间则放弃获取锁。<br>3.释放锁的时候,通过UUID判断是不是当前持有的锁,若时该锁,则执行delete进行锁释放。</p></blockquote><p>具体实现代码如下:</p><figure class="highlight java"><table><tr><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="class"><span class="keyword">class</span> <span class="title">DistributedLock</span> </span>{</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> JedisPool jedisPool;</span><br><span class="line"> <span class="keyword">private</span> <span class="keyword">final</span> <span class="keyword">static</span> String KEY_PREF = <span class="string">"lock:"</span>; <span class="comment">// 锁的前缀</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="title">DistributedLock</span><span class="params">(JedisPool jedisPool)</span> </span>{</span><br><span class="line"> <span class="keyword">this</span>.jedisPool = jedisPool;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 加锁</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> lockName String 锁的名称(key)</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> acquireTimeout long 获取超时时间</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@param</span> timeout long 锁的超时时间</span></span><br><span class="line"><span class="comment"> * <span class="doctag">@return</span> 锁标识</span></span><br><span class="line"><span class="comment"> */</span></span><br><span class="line"> <span class="function"><span class="keyword">public</span> String <span class="title">lockWithTimeout</span><span class="params">(String lockName, <span class="keyword">long</span> acquireTimeout, <span class="keyword">long</span> timeout)</span> </span>{</span><br><span class="line"> Jedis conn = <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> <span class="comment">// 获取连接</span></span><br><span class="line"> conn = jedisPool.getResource();</span><br><span class="line"> <span class="comment">// 随机生成一个value</span></span><br><span class="line"> String identifier = UUID.randomUUID().toString();</span><br><span class="line"> <span class="comment">// 锁名,即 key值</span></span><br><span class="line"> String lockKey = KEY_PREF + lockName;</span><br><span class="line"> <span class="comment">// 超时时间, 上锁后超过此时间则自动释放锁</span></span><br><span class="line"> <span class="keyword">int</span> lockExpire = (<span class="keyword">int</span>) (timeout / <span class="number">1000</span>);</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 获取锁的超时时间,超过这个时间则放弃获取锁</span></span><br><span class="line"> <span class="keyword">long</span> end = System.currentTimeMillis() + acquireTimeout;</span><br><span class="line"> <span class="keyword">while</span> (System.currentTimeMillis() < end) {</span><br><span class="line"> <span class="keyword">if</span> (conn.setnx(lockKey, identifier) == <span class="number">1</span>) {</span><br><span class="line"> conn.expire(lockKey, lockExpire);</span><br><span class="line"> <span class="comment">// 返回value值,用于释放锁时间确认</span></span><br><span class="line"> <span class="keyword">return</span> identifier;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="comment">// 返回-1代表key没有设置超时时间,为key设置一个超时时间</span></span><br><span class="line"> <span class="keyword">if</span> (conn.ttl(lockKey) == -<span class="number">1</span>) {</span><br><span class="line"> conn.expire(lockKey, lockExpire);</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> Thread.sleep(<span class="number">10</span>);</span><br><span class="line"> } <span class="keyword">catch</span> (InterruptedException ie) {</span><br><span class="line"> Thread.currentThread().interrupt();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (JedisException e) {</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (conn != <span class="keyword">null</span>) {</span><br><span class="line"> conn.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">null</span>;</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><figure class="highlight java"><table><tr><td class="code"><pre><span class="line">**</span><br><span class="line"> * 释放锁</span><br><span class="line"> *</span><br><span class="line"> * <span class="meta">@param</span> lockName String 锁key</span><br><span class="line"> * <span class="meta">@param</span> identifier String 释放锁的标识</span><br><span class="line"> * <span class="meta">@return</span> <span class="keyword">boolean</span></span><br><span class="line"> */</span><br><span class="line"> <span class="function"><span class="keyword">public</span> <span class="keyword">boolean</span> <span class="title">releaseLock</span><span class="params">(String lockName, String identifier)</span> </span>{</span><br><span class="line"> Jedis conn = <span class="keyword">null</span>;</span><br><span class="line"> String lockKey = KEY_PREF + lockName;</span><br><span class="line"> <span class="keyword">boolean</span> retFlag = <span class="keyword">false</span>;</span><br><span class="line"> <span class="keyword">try</span> {</span><br><span class="line"> conn = jedisPool.getResource();</span><br><span class="line"> <span class="keyword">while</span> (<span class="keyword">true</span>) {</span><br><span class="line"> <span class="comment">// 监视lock, 准备开始事务</span></span><br><span class="line"> conn.watch(lockKey);</span><br><span class="line"> <span class="comment">// 通过前面返回的value值判断是不是该锁,若时该锁,则删除释放锁</span></span><br><span class="line"> <span class="keyword">if</span> (identifier.equals(conn.get(lockKey))) {</span><br><span class="line"> <span class="comment">// 开启redis事务</span></span><br><span class="line"> Transaction transaction = conn.multi();</span><br><span class="line"> transaction.del(lockKey);</span><br><span class="line"> <span class="comment">// 关闭redis事务</span></span><br><span class="line"> List<Object> results = transaction.exec();</span><br><span class="line"> <span class="keyword">if</span> (results == <span class="keyword">null</span>) <span class="keyword">continue</span>;</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> retFlag = <span class="keyword">true</span>;</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> conn.unwatch();</span><br><span class="line"> <span class="keyword">break</span>;</span><br><span class="line"> }</span><br><span class="line"> } <span class="keyword">catch</span> (Exception e) {</span><br><span class="line"> e.printStackTrace();</span><br><span class="line"> } <span class="keyword">finally</span> {</span><br><span class="line"> <span class="keyword">if</span> (conn != <span class="keyword">null</span>) {</span><br><span class="line"> conn.close();</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> <span class="keyword">return</span> retFlag;</span><br><span class="line"> }</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="基于Zookeeper的实现方式"><a href="#基于Zookeeper的实现方式" class="headerlink" title="基于Zookeeper的实现方式"></a>基于Zookeeper的实现方式</h5><p>基于Zookeeper临时有序节点同样可以实现分布式锁。</p><p>大致思想为:</p><blockquote><p>每个客户端对某个方法加锁时,在zookeeper上的该方法对应的指定节点目录下,生成一个唯一的瞬时有序节点。</p></blockquote><blockquote><p>判断是否获取锁的方式很简单,只需要判断有序节点中序号最小的一个。如果获取到比自己小的兄弟节点不存在,则说明当前线程顺序号最小,获得锁。</p></blockquote><blockquote><p>如果判断自己不是那最小的一个节点,则设置监听比自己次小的节点;</p></blockquote><blockquote><p>如果已处理完成,则删除自己的节点。</p></blockquote><p>优点:</p><blockquote><p>具备高可用、可重入、阻塞锁特性、可解决失效死锁问题。</p></blockquote><p>缺点:</p><blockquote><p>因为需要频繁的创建和删除节点,性能上不如Redis方式。</p></blockquote><h5 id="三种方案的比较"><a href="#三种方案的比较" class="headerlink" title="三种方案的比较"></a>三种方案的比较</h5><p>从理解的难易程度(从低到高):</p><blockquote><p>数据库 > 缓存 > Zookeeper</p></blockquote><p>从实现的复杂性角度(从低到高):</p><blockquote><p>Zookeeper >= 缓存 > 数据库</p></blockquote><p>从性能角度(从高到低):</p><blockquote><p>缓存 > Zookeeper >= 数据库</p></blockquote><p>从可靠性角度(从高到低):</p><blockquote><p>Zookeeper > 缓存 > 数据库</p></blockquote>]]></content>
<categories>
<category> 分布式 </category>
</categories>
<tags>
<tag> 分布式 </tag>
</tags>
</entry>
<entry>
<title>Rainbow</title>
<link href="/about/index.html"/>
<url>/about/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><h2 id="个人介绍"><a href="#个人介绍" class="headerlink" title="个人介绍"></a>个人介绍</h2><p><code>Design, to build a simpler world</code></p><blockquote><p> 工作刚满一年,Java开发。</p><p>对一切不了解的事物都感兴趣,做这个博客也是觉得好玩。</p><p>人生三大乐趣:音乐,电影,羽毛球</p></blockquote><h2 id="联系方式"><a href="#联系方式" class="headerlink" title="联系方式"></a>联系方式</h2><p>GitHub: <a href="https://github.com/liuHongJie1217" target="_blank" rel="noopener">rainbow</a></p><p>微博:<a href="https://weibo.com/7336185047/profile?topnav=1&wvr=6&is_all=1" target="_blank" rel="noopener">存在即合理云云</a></p><p>邮箱:<a href="mailto:840474510@qq.com" target="_blank" rel="noopener">840474510@qq.com</a></p>]]></content>
</entry>
<entry>
<title>分类</title>
<link href="/categories/index.html"/>
<url>/categories/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script>]]></content>
</entry>
<entry>
<title>友情链接</title>
<link href="/link/index.html"/>
<url>/link/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script>]]></content>
</entry>
<entry>
<title>音乐分享</title>
<link href="/music/index.html"/>
<url>/music/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><!-- <iframe align="middle" frameborder="no" border="0" marginwidth="0" marginheight="0" width=330 height=450 src="www.xiami.com/collect/796705375"></iframe> --> <div id="aplayer-GhAsDHuw" class="aplayer aplayer-tag-marker meting-tag-marker" data-id="796705375" data-server="xiami" data-type="playlist" data-mode="circulation" data-autoplay="false" data-mutex="true" data-listmaxheight="340px" data-preload="auto" data-theme="#FF4081" ></div>]]></content>
</entry>
<entry>
<title>标签</title>
<link href="/tags/index.html"/>
<url>/tags/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script>]]></content>
</entry>
<entry>
<title>视频分享</title>
<link href="/video/index.html"/>
<url>/video/index.html</url>
<content type="html"><![CDATA[<link rel="stylesheet" class="aplayer-secondary-style-marker" href="/assets/css/APlayer.min.css"><script src="/assets/js/APlayer.min.js" class="aplayer-secondary-script-marker"></script><script class="meting-secondary-script-marker" src="/assets/js/Meting.min.js"></script><!-- <div id="dplayer0" class="dplayer hexo-tag-dplayer-mark" style="margin-bottom: 20px;"></div><script>(function(){var player = new DPlayer({"container":document.getElementById("dplayer0"),"theme":"#FADFA3","loop":true,"video":{"url":"https://www.xiami.com/mv/K6YhqC","pic":"https://i.loli.net/2019/11/13/LFB9uVNzpSbH82a.jpg"}});window.dplayers||(window.dplayers=[]);window.dplayers.push(player);})()</script> --><div id="dplayer1" class="dplayer hexo-tag-dplayer-mark" style="margin-bottom: 20px;"></div><script>(function(){var player = new DPlayer({"container":document.getElementById("dplayer1"),"theme":"#FADFA3","loop":true,"video":{"url":"/img/video.mp4","pic":"https://i.loli.net/2019/11/13/LFB9uVNzpSbH82a.jpg"}});window.dplayers||(window.dplayers=[]);window.dplayers.push(player);})()</script>]]></content>
</entry>
</search>
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。