SpringBoot+Sharding-JDBC操作分库分表

Sharding-JDBC操作水平分表

本文为CSDN博主「1-只小猴子」的原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接及本声明。

一、搭建环境

1.基础环境:SpringBoot2.2.1 + MybatisPlus + Sharding-JDBC + Druid连接池
2.创建SpringBoot工程
image
3.修改SpringBoot项目版本为2.2.1
4.引入相关依赖

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
 
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.junit.vintage</groupId>
                    <artifactId>junit-vintage-engine</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
 
        <!-- Druid连接池 -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>druid</artifactId>
            <version>1.1.20</version>
        </dependency>
 
        <!-- Mysql驱动依赖 -->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
        </dependency>
 
        <!-- MybatisPlus -->
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
            <version>3.0.5</version>
        </dependency>
 
        <!-- Sharding-JDBC -->
        <dependency>
            <groupId>org.apache.shardingsphere</groupId>
            <artifactId>sharding-jdbc-spring-boot-starter</artifactId>
            <version>4.0.0-RC1</version>
        </dependency>
 
        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
        </dependency>

    </dependencies>

按照水平分表的方式创建数据库和数据库表

二、按照水平分表的方式创建数据库和数据库表

1.创建数据库goods_db;
2.在goods_db中创建表goods_1、goods_2;
3.约定规则,如果添加商品id是偶数把数据加入goods_1,如果是偶数把数据加入goods_2;
结构展示
image

三、编写代码

创建Goods实体类

package com.ws.shardingjdbcdemo.pojo;
 
import lombok.Data;
 
//@Data注解是lombok的注解,简化实体类编写,自动生成get/set以及toString等方法
@Data
public class Goods {
    private Long gid;
    private String gname;
    private Long userId;
    private String gstatus;
}

创建GoodsMapper

package com.ws.shardingjdbcdemo.mapper;
 
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.ws.shardingjdbcdemo.pojo.Goods;
import org.springframework.stereotype.Repository;
 
@Repository
public interface GoodsMapper extends BaseMapper<Goods> {
 
}

启动类配置扫描Mapper包


package com.ws.shardingjdbcdemo;
 
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
 
@SpringBootApplication
//扫描包
@MapperScan("com.ws.shardingjdbcdemo.mapper")
public class ShardingjdbcdemoApplication {
 
    public static void main(String[] args) {
        SpringApplication.run(ShardingjdbcdemoApplication.class, args);
    }
 
}

四、application.properties配置Sharding-JDBC

# 配置Sharding-JDBC的分片策略
# 配置数据源,给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1
 
# 配置数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
# 由于上面配置数据源只有g1因此下面只配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
 
# 配置表的分布,表的策略
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g1.goods_$->{1..2}
 
# 指定goods表 主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
 
# 指定分片策略 约定gid值是偶数添加到goods_1表,如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
 
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

五、编写测试代码


package com.ws.shardingjdbcdemo;
 
import com.ws.shardingjdbcdemo.mapper.GoodsMapper;
import com.ws.shardingjdbcdemo.pojo.Goods;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
 
@SpringBootTest
class ShardingjdbcdemoApplicationTests {
 
    @Autowired
    GoodsMapper goodsMapper;
 
    @Test
    void addGoods() {
        Goods good = new Goods();
        good.setGname("小米手机");
        good.setUserId(100L);
        good.setGstatus("已发布");
        goodsMapper.insert(good);
    }
 
}

image
启动测试方法,果然报错了:
image
大概意思就是说,不允许有两个相同名称的Bean。

解决方案:

spring.main.allow-bean-definition-overriding=true

将此配置在application.properties中

# 配置Sharding-JDBC的分片策略
# 配置数据源,给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1
 
# 配置允许后面的Bean覆盖前面名称重复的Bean
spring.main.allow-bean-definition-overriding=true
 
# 配置数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
# 由于上面配置数据源只有g1因此下面只配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
 
# 配置表的分布,表的策略
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g1.goods_$->{1..2}
 
# 指定goods表 主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
 
# 指定分片策略 约定gid值是偶数添加到goods_1表,如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
 
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

再次启动测试方法成功插入数据并在控制台打印出SQL语句:
image
接下来测试一下是否真正按照奇数偶数来分表插入了。

修改addGoods()方法的代码:

    @Test
    void addGoods() {
        for (int i = 0; i < 10; i++){
            Goods good = new Goods();
            good.setGname("小米手机" + i);
            good.setUserId(100L);
            good.setGstatus("已发布");
            goodsMapper.insert(good);
        }
    }
	

image

Sharding-JDBC操作水平分库+水平分表

  1. 创建两个数据库goods_db_1和goods_db_2,每个数据库中均包含两个表goods_1和goods_2,goods_1和goods_2和上述分表的结构一样。
    image
  2. 配置application.properties
# 配置Sharding-JDBC的分片策略
# 配置数据源,给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2
 
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
 
# 配置数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
 
# 配置数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
 
# 配置数据库的分布,表的分布
# m1:goods_1 goods_2; m2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
 
# 指定goods表 主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
 
# 指定数据库分片策略 约定user_id值是偶数添加到goods_db_1中,奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
 
# 指定表分片策略 约定gid值是偶数添加到goods_1表,如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
 
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true
3. 编写测试方法addGoods02()
 @Test
void addGoods02(){
    Goods good = new Goods();
    good.setGname("华为手机");
    good.setUserId(100L);
    good.setGstatus("已发布");
    goodsMapper.insert(good);
}

4. 执行addGoods02()方法

由于我们的user_id设置为100L,为偶数,根据我们的初衷,偶数存入goods_db_1中。
image

Sharding-JDBC操作垂直分库+水平分库+水平分表

背景描述:

  在  分库分表&分库分表利器 文章中了解到什么是垂直分库分表、水平分库分表,为什么分库分表;在咱们实际项目中,垂直分库后会有多个数据库,每个数据库中有属于自己的表和公共表(操作公共表在本节后面会讲述)。

  上面章节我们做到了用Sharding-JDBC操作水平分库分表,在以上基础上,我们还有一个数据库user_db,里面有一张t_user表,用来存放用户信息,这是在咱们一个项目中有多个数据库,每个数据库又有不同的表。 

数据库结构及t_user表结构如下:

image

需求分析

在我们项目中有以上这么多数据库和表,当我们操作t_user表时,快速准确操作到userdb数据库 > t_user表。

二、实现需求

  1. 配置application.properties文件

在上面水平分库分表的基础上编写关于user_db和t_user表的配置。

二、实现需求

  1. 配置application.properties文件

    在上面水平分库分表的基础上编写关于user_db和t_user表的配置。

# 配置Sharding-JDBC的分片策略
# 配置数据源,给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2,u0
 
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
 
# 配置g1数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
 
# 配置g2数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
 
# 配置u0数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 u0配置u0.type,u0.driver-class-name,u0.url,u0.username,u0.password
spring.shardingsphere.datasource.u0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.u0.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.u0.url=jdbc:mysql://localhost:3306/user_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.u0.username=root
spring.shardingsphere.datasource.u0.password=123456
 
#配置垂直分库t_user的策略
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=u0.t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=user_id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=user_id
#由于只有一张表,因此在此直接写表明,不需要像水平分多个表那样写策略
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
 
# 配置数据库的分布,表的分布
# g1:goods_1 goods_2; g2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
 
# 指定goods表 主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
 
# 指定数据库分片策略 约定user_id值是偶数添加到goods_db_1中,奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
 
# 指定表分片策略 约定gid值是偶数添加到goods_1表,如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
 
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true
  1. 编写代码
    image

User.java

package com.ws.shardingjdbcdemo.pojo;
 
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
 
@Data
@TableName("t_user")
public class User {
    private Long userId;
    private String username;
    private String ustatus;
}

UserMapper.java

package com.ws.shardingjdbcdemo.mapper;
 
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.ws.shardingjdbcdemo.pojo.User;
import org.springframework.stereotype.Repository;
 
@Repository
public interface UserMapper extends BaseMapper<User> {
 
}

Tests.java

    @Test
    void addUser(){
        User user = new User();
        user.setUsername("琳妹妹");
        user.setUstatus("0");
        userMapper.insert(user);
    }
 
    @Test
    void getUser(){
        QueryWrapper<User> queryWrapper = new QueryWrapper<>();
        //此处请填写自己程序生成的ID
        queryWrapper.eq("user_id",100L);
        User good = userMapper.selectOne(queryWrapper);
        System.out.println(good.toString());
    }

三、测试及结果

image
image

Sharding-JDBC操作公共表

背景描述:

    在项目中一般会有一些表的内容是固定的,或者说是很少修改的表,但是又经常跟这些表关联查询,例如一些状态信息。一般在我们项目中这种表会存放在我们各个数据库,所以称为公共表。

    在上面章节我们使用Sharding-JDBC操作了垂直分库+水平分库+水平分表,假设现在在我们的各个数据库中均有一张公共表t_dict。

数据库&表结构:
image
需求:

操作公共表时,例如增加和删除操作时,会修改所有数据库中的这张表。

实现:

1、在上章节基础上编写配置 application.properties(配置公共表部分)

# 配置Sharding-JDBC的分片策略
# 配置数据源,给数据源起名g1,g2...此处可配置多数据源
spring.shardingsphere.datasource.names=g1,g2,u0
 
# 配置允许一个实体类映射多张表
spring.main.allow-bean-definition-overriding=true
 
# 配置g1数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g1配置g1.type,g1.driver-class-name,g1.url,g1.username,g1.password
spring.shardingsphere.datasource.g1.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g1.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g1.url=jdbc:mysql://localhost:3306/goods_db_1?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g1.username=root
spring.shardingsphere.datasource.g1.password=123456
 
# 配置g2数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 g2配置g2.type,g2.driver-class-name,g2.url,g2.username,g2.password
spring.shardingsphere.datasource.g2.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.g2.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.g2.url=jdbc:mysql://localhost:3306/goods_db_2?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.g2.username=root
spring.shardingsphere.datasource.g2.password=123456
 
# 配置u0数据源具体内容————————包含  连接池,  驱动,             地址,   用户名,    密码
#                                 u0配置u0.type,u0.driver-class-name,u0.url,u0.username,u0.password
spring.shardingsphere.datasource.u0.type=com.alibaba.druid.pool.DruidDataSource
spring.shardingsphere.datasource.u0.driver-class-name=com.mysql.cj.jdbc.Driver
spring.shardingsphere.datasource.u0.url=jdbc:mysql://localhost:3306/user_db?characterEncoding=utf-8&useUnicode=true&useSSL=false&serverTimezone=UTC
spring.shardingsphere.datasource.u0.username=root
spring.shardingsphere.datasource.u0.password=123456
 
# 配置公共表
spring.shardingsphere.sharding.broadcast-tables=t_dict
# 配置公共表ID及生成策略
spring.shardingsphere.sharding.tables.t_dict.key-generator.column=dict_id
spring.shardingsphere.sharding.tables.t_dict.key-generator.type=SNOWFLAKE
 
#配置垂直分库t_user的策略
spring.shardingsphere.sharding.tables.t_user.actual-data-nodes=u0.t_user
spring.shardingsphere.sharding.tables.t_user.key-generator.column=user_id
spring.shardingsphere.sharding.tables.t_user.key-generator.type=SNOWFLAKE
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.sharding-column=user_id
#由于只有一张表,因此在此直接写表明,不需要像水平分多个表那样写策略
spring.shardingsphere.sharding.tables.t_user.table-strategy.inline.algorithm-expression=t_user
 
# 配置数据库的分布,表的分布
# g1:goods_1 goods_2; g2:goods_1,goods_2;
spring.shardingsphere.sharding.tables.goods.actual-data-nodes=g$->{1..2}.goods_$->{1..2}
 
# 指定goods表 主键gid 生成策略为 SNOWFLAKE
spring.shardingsphere.sharding.tables.goods.key-generator.column=gid
spring.shardingsphere.sharding.tables.goods.key-generator.type=SNOWFLAKE
 
# 指定数据库分片策略 约定user_id值是偶数添加到goods_db_1中,奇数添加到goods_db_2中
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.sharding-column=user_id
spring.shardingsphere.sharding.tables.goods.database-strategy.inline.algorithm-expression=g$->{user_id % 2 + 1}
 
# 指定表分片策略 约定gid值是偶数添加到goods_1表,如果gid是奇数添加到goods_2表
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.sharding-column=gid
spring.shardingsphere.sharding.tables.goods.table-strategy.inline.algorithm-expression=goods_$->{gid % 2 + 1}
 
# 打开sql输出日志
spring.shardingsphere.props.sql.show=true

2、编写代码

MyDict.java

package com.ws.shardingjdbcdemo.pojo;
 
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
 
@Data
@TableName("t_dict")
public class MyDict {
    private Long dictId;
    private String dictName;
    private String dictCode;
}

MyDictMapper.java


package com.ws.shardingjdbcdemo.mapper;
 
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import com.ws.shardingjdbcdemo.pojo.MyDict;
import org.springframework.stereotype.Repository;
 
@Repository
public interface MyDictMapper extends BaseMapper<MyDict> {
 
}

Tests.java

    /**
     * 下面是公共表测试方法
     */
    @Test
    void addDict(){
        MyDict myDict = new MyDict();
        myDict.setDictName("已启用");
        myDict.setDictCode("1");
        myDictMapper.insert(myDict);
    }
 
    @Test
    void deleteDict(){
        QueryWrapper<MyDict> wrapper = new QueryWrapper<>();
        wrapper.eq("dict_id","");
        myDictMapper.delete(wrapper);
    }

3、测试

addDict()方法:
image
以上就是通过Apache-ShardingSphere的Sharding-JDBC对分库分表以及公共表的操作

posted @ 2023-03-12 16:11  嗨^_^  阅读(225)  评论(0)    收藏  举报