从外部存储库动态加载 Spring 属性

借助 Spring Framework 的 Property Sources,开发人员可以将其应用程序配置为从外部存储库动态加载配置 - 使用可编程 API 保护密钥库、数据库或任何其他源。
外部化配置是软件开发中的一种有价值的模式,可实现集中配置管理、运行时灵活性和更高的安全性。有关更多信息,请参阅 microservices.io 上的外部化配置。

在本文中,我们从 Spring Framework 扩展了EnvironmentPostProcessorEnumerablePropertySource类来实现自定义的动态属性加载器 - 我们的示例将使用 Oracle 数据库作为属性存储库,但此模式适用于任何外部化配置(无论是否为数据库)。

您可以在 GitHub 上找到使用 Oracle 数据库的属性源示例包括其依赖项。如果您对更多外部配置感兴趣,请参阅Spring Cloud Config以获取分布式系统中的配置示例。

  先决条件

跟随该示例需要对 Java 和 Spring 有一定的了解。

  •   Java 21+、Maven
  • Docker 兼容环境,内存约 8 GB

我们的自定义属性源的属性?

与其他组件一样,属性源可能需要自己的属性和自定义配置。
为了解决这个问题,我们定义了一个 Spring 配置类,它接受数据库表列表作为属性源,以及一个可选的属性刷新间隔以从数据库重新加载属性。

请参阅DatabaseProperties.java :

package com.example;

import java.time.Duration;
import java.util.List;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

@ConfigurationProperties(prefix = DatabaseProperties.PREFIX)
@Component
public class DatabaseProperties {
    public static final String PREFIX = "database";

    // Used to refresh properties on an interval
    private Duration propertyRefreshInterval;

    // We may have 0 or more property sources
    private List<PropertySource> propertySources;

    // A property source is a database table
    public static class PropertySource {
        private String table;

        public String getTable() {
            return table;
        }

        public void setTable(String table) {
            this.table = table;
        }
    }

    public Duration getPropertyRefreshInterval() {
        return propertyRefreshInterval;
    }

    public void setPropertyRefreshInterval(Duration propertyRefreshInterval) {
        this.propertyRefreshInterval = propertyRefreshInterval;
    }

    public List<PropertySource> getPropertySources() {
        return propertySources;
    }

    public void setPropertySources(List<PropertySource> propertySources) {
        this.propertySources = propertySources;
    }
}

因为我们的属性加载器使用 JDBC,所以我们需要使用 Spring 配置应用程序数据源。对于 Oracle 数据库的示例,我们使用以下 application.yaml,其中 JDBC 用户名、密码和 URL 是从环境变量中替换的。

我们还使用名为spring_property的表和 1000 毫秒的刷新间隔设置 DatabaseProperties - 这些属性将在应用程序启动期间用于从数据库加载属性。

spring:
  datasource:
    username: ${USERNAME}
    password: ${PASSWORD}
    url: ${JDBC_URL}

    # Set these to use UCP over Hikari.
    driver-class-name: oracle.jdbc.OracleDriver
    type: oracle.ucp.jdbc.PoolDataSourceImpl
    oracleucp:
      initial-pool-size: 1
      min-pool-size: 1
      max-pool-size: 30
      connection-pool-name: UCPSampleApplication
      connection-factory-class-name: oracle.jdbc.pool.OracleDataSource


# Load properties from the spring_property database table every 1000ms
database:
  property-sources:
    - table: spring_property
  property-refresh-interval: 1000ms

实现自定义属性加载器和属性源

我们实现DatabasePropertyLoader类来使用Spring的JdbcTemplate从数据库加载数据。每个属性都会在应用程序启动时由加载程序查询并存储在内存映射中。如果提供了刷新间隔,计时器会按该间隔重新加载属性。

请注意Property POJO的使用,用于将数据库行映射到我们的属性加载器可用的对象。

package com.example;

import java.time.Duration;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Timer;
import java.util.TimerTask;

import org.springframework.jdbc.core.JdbcTemplate;

/**
 * A utility class responsible for loading and periodically refreshing database properties from a specified table.
 * It uses the Spring JDBC Template to execute queries against the database and stores the loaded properties in an internal map.
 *
 * @author Anders Swanson
 */
public class DatabasePropertyLoader implements AutoCloseable {
    private static Timer timer;

    private final String table;
    private final JdbcTemplate jdbcTemplate;
    private Map<String, String> properties = new LinkedHashMap<>();

    /**
     * Constructs a new instance of DatabasePropertyLoader that loads and periodically refreshes
     * database properties from the specified table using the provided JdbcTemplate.
     *
     * @param table   the name of the database table containing the properties
     * @param jdbcTemplate the Spring JDBC Template used to execute queries against the database
     * @param refresh the duration between automatic refreshes of the properties, or 0 ms to disable property refresh.
     *                The default refresh rate is 10 minutes if not specified.
     */
    public DatabasePropertyLoader(String table, JdbcTemplate jdbcTemplate, Duration refresh) {
        this.table = table;
        this.jdbcTemplate = jdbcTemplate;
        reload();
        long refreshMillis = Optional.ofNullable(refresh)
                .orElse(Duration.ofMinutes(10))
                .toMillis();
        if (refreshMillis > 0) {
            synchronized (DatabasePropertyLoader.class) {
                if (timer == null) {
                    timer = new Timer(true);
                    timer.scheduleAtFixedRate(new TimerTask() {
                        @Override
                        public void run() {
                            reload();
                        }
                    }, refreshMillis, refreshMillis);
                }
            }
        }
    }

    boolean containsProperty(String key) {
        return properties.containsKey(key);
    }

    Object getProperty(String key) {
        return properties.get(key);
    }

    String[] getPropertyNames() {
        return properties.keySet().toArray(String[]::new);
    }

    /**
     * Reloads the database properties from the specified table into memory.
     * This method executes a SQL query to retrieve all key-value pairs from the table,
     * then updates the internal map of properties with the retrieved values.
     */
    private void reload() {
        String query = "select * from %s".formatted(table);
        List<Property> result = jdbcTemplate.query(query, (rs, rowNum)
                -> new Property(rs.getString("key"), rs.getString("value")));
        properties = new HashMap<>(properties.size());
        for (Property property : result) {
            properties.put(property.key(), property.value());
        }
    }

    @Override
    public void close() throws Exception {
        synchronized (DatabasePropertyLoader.class) {
            if (timer != null) {
                timer.cancel();
                timer = null;
            }
        }
    }
}

我们现在实现DatabasePropertySource类,提供一个为给定 DatabasePropertyLoader 实例扩展EnumerablePropertySource的包装器。扩展 EnumerablePropertySource 或其他属性源驱动类对于使用 Spring 注册属性源至关重要。

DatabasePropertySource 的每个实例都配置有 DatabasePropertyLoader,以便于访问已从特定数据库表加载到内存中的属性。

package com.example;

import org.springframework.core.env.EnumerablePropertySource;

/**
 * A custom {@link org.springframework.core.env.PropertySource} implementation that retrieves properties from a database.
 * This class extends {@link EnumerablePropertySource} and delegates property access to an underlying
 * {@link DatabasePropertyLoader}.
 *
 * @see DatabasePropertyLoader
 */
public class DatabasePropertySource extends EnumerablePropertySource<DatabasePropertyLoader> {
    public DatabasePropertySource(String name, DatabasePropertyLoader source) {
        super(name, source);
    }

    @Override
    public String[] getPropertyNames() {
        return source.getPropertyNames();
    }

    @Override
    public Object getProperty(String name) {
        return source.getProperty(name);
    }

    @Override
    public boolean containsProperty(String name) {
        return source.containsProperty(name);
    }
}

注册环境后处理器

我们现在可以扩展EnvironmentPostProcessor 类,以便Spring 在应用程序启动期间调用我们的自定义数据库属性源。

如果您之前没有使用过EnvironmentPostProcessor,它会被扩展为在应用程序启动时加载所有bean之前修改Spring 环境。如果您在此阶段需要访问任何 Spring 组件,则需要使用 Binder 类自行实例化它们,就像我们下面使用 DataSource 和 JdbcTemplate 所做的那样。

package com.example;

import javax.sql.DataSource;
import java.time.Duration;
import java.util.List;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor;
import org.springframework.boot.context.properties.bind.Bindable;
import org.springframework.boot.context.properties.bind.Binder;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.core.Ordered;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.util.ClassUtils;

import static org.springframework.core.env.StandardEnvironment.SYSTEM_ENVIRONMENT_PROPERTY_SOURCE_NAME;

/**
 * A Spring Boot {@link EnvironmentPostProcessor} implementation that loads database properties
 * after the application has been initialized but before it starts up.
 */
public class DatabaseEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered {
    /**
     * Post-processes the Spring environment by loading database properties and adding them as property sources.
     * This method is called after the application has been initialized but before it starts up.
     * It creates a JdbcTemplate from the Spring environment, loads the property source properties,
     * and adds the database property sources to the collection of Spring property sources.
     *
     * @param environment the Spring environment to be processed
     * @param application the Spring application instance
     */
    @Override
    public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
        // Create a JdbcTemplate from the Spring environment
        Binder binder = Binder.get(environment);
        DataSourceProperties dataSourceProperties = binder.bind("spring.datasource", Bindable.of(DataSourceProperties.class))
                .orElse(new DataSourceProperties());
        DataSource dataSource = dataSourceProperties.initializeDataSourceBuilder()
                .build();
        JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);

        // Load the property source properties from the Spring environment
        DatabaseProperties databaseProperties = binder.bind(
                DatabaseProperties.PREFIX,
                Bindable.of(DatabaseProperties.class)
        ).orElse(new DatabaseProperties());
        List<DatabaseProperties.PropertySource> databasePropertySources = databaseProperties.getPropertySources();
        Duration refreshInterval = databaseProperties.getPropertyRefreshInterval();

        MutablePropertySources propertySources = environment.getPropertySources();
        for (DatabaseProperties.PropertySource source : databasePropertySources) {
            DatabasePropertyLoader propertyLoader = new DatabasePropertyLoader(source.getTable(), jdbcTemplate, refreshInterval);
            DatabasePropertySource propertySource = new DatabasePropertySource(source.getTable(), propertyLoader);

            // Add the database property source to the collection of Spring property sources
            if (propertySources.contains(SYSTEM_ENVIRONMENT_PROPERTY_SOURCE_NAME)) {
                propertySources.addAfter(SYSTEM_ENVIRONMENT_PROPERTY_SOURCE_NAME, propertySource);
            } else {
                propertySources.addFirst(propertySource);
            }
        }
    }

    @Override
    public int getOrder() {
        return ConfigDataEnvironmentPostProcessor.ORDER + 1;
    }
}

重要提示:为了让 Spring 调用任何 EnvironmentPostProcessor 扩展,这些扩展必须在应用程序的resources/META-INF/spring.factories文件中注册。对于我们的环境后处理器实现,它看起来像这样:

org.springframework.boot.env.EnvironmentPostProcessor=com.example.DatabaseEnvironmentPostProcessor

添加 PropertyService 和入口点

为了帮助测试我们的属性源实现是否正常工作,我们实现了一个PropertyService类,该类使用带有 Spring Value注释的数据库属性 - 这将演示该属性是由 Spring 从数据库加载的,并且可以通过 @Value 访问。

package com.example;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Component;

@Component
public class PropertyService {
    private static final String PROPERTY_TABLE = "spring_property";
    private static final String UPDATE_PROPERTY = """
            update %s set value = ? where key = ?
            """.formatted(PROPERTY_TABLE);
    private final JdbcTemplate jdbcTemplate;

    /**
     * The value of 'property1' is dynamically loaded during
     * startup, using our database property source implementation.
     */
    @Value("${property1}")
    private String property1;

    public PropertyService(JdbcTemplate jdbcTemplate) {
        this.jdbcTemplate = jdbcTemplate;
    }

    /**
     * Used to demonstrate dynamic property loading, using the value of
     * 'property1' loaded from the database property source.
     * @return the current value of property1.
     */
    public String getProperty1() {
        return property1;
    }

    /**
     * Updates an existing property in the database.
     *
     * @param property the updated property object containing the new value and key
     */
    public void updateProperty(Property property) {
        jdbcTemplate.update(UPDATE_PROPERTY, property.value(), property.key());
    }
}

最后,我们添加一个Spring 应用程序主类,这将是任何 Spring Boot 应用程序的标准。

package com.example;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SampleApp {
    public static void main(String[] args) {
        SpringApplication.run(SampleApp.class, args);
    }
}

  测试样品

我们实现了一个自定义属性源,但现在我们需要测试它 - DatabasePropertySourceTest.java实现属性加载器的端到端测试,使用 Testcontainers 实例化容器化的 Oracle 数据库以保存和加载属性。

测试实现验证属性是否已从数据库加载到 PropertyService,然后更新属性并确保其已刷新。

package com.example;

import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;
import java.time.Duration;

import oracle.jdbc.pool.OracleDataSource;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.support.DefaultSingletonBeanRegistry;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ConfigurableApplicationContext;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.oracle.OracleContainer;

import static org.assertj.core.api.Assertions.assertThat;

@Testcontainers
@SpringBootTest
public class DatabasePropertySourceTest {
    /**
     * Use a containerized Oracle Database instance to test the Database property source.
     */
    static OracleContainer oracleContainer = new OracleContainer("gvenzl/oracle-free:23.5-slim-faststart")
            .withStartupTimeout(Duration.ofMinutes(2))
            .withUsername("testuser")
            .withPassword(("testpwd"));

    /**
     * Dynamically configure Spring Boot properties to use the Testcontainers database.
     */
    @BeforeAll
    static void setUp() throws SQLException {
        oracleContainer.start();
        System.setProperty("JDBC_URL", oracleContainer.getJdbcUrl());
        System.setProperty("USERNAME", oracleContainer.getUsername());
        System.setProperty("PASSWORD", oracleContainer.getPassword());

        // Configure a datasource for the Oracle Database container.
        OracleDataSource dataSource = new OracleDataSource();
        dataSource.setUser(oracleContainer.getUsername());
        dataSource.setPassword(oracleContainer.getPassword());
        dataSource.setURL(oracleContainer.getJdbcUrl());
        // Create the property source table, and populate it with some
        // initial data
        try (Connection conn = dataSource.getConnection();
             Statement stmt = conn.createStatement()) {
            stmt.executeUpdate("""
                create table spring_property (
                key varchar2(255) primary key not null ,
                value varchar2(255) not null
            )""");
            stmt.executeUpdate(" insert into spring_property (key, value) values ('property1', 'initial value')");
        }
    }

    @Autowired
    PropertyService propertyService;

    @Autowired
    DatabaseProperties databaseProperties;

    @Autowired
    ApplicationContext applicationContext;

    @Test
    void propertySourceTest() throws InterruptedException {
        System.out.println("Starting Property Source Test");

        String property1 = propertyService.getProperty1();
        assertThat(property1).isNotNull();
        assertThat(property1).isEqualTo("initial value");
        System.out.println("Value of 'property1': " + property1);


        System.out.println("Updating Property 'property1'");
        propertyService.updateProperty(new Property("property1", "updated"));

        // Wait for property to be refreshed
        Thread.sleep(databaseProperties.getPropertyRefreshInterval().plusMillis(200));

        System.out.println("Reloading PropertyService Bean");
        ConfigurableApplicationContext configContext = (ConfigurableApplicationContext) applicationContext;
        DefaultSingletonBeanRegistry registry = (DefaultSingletonBeanRegistry) configContext.getBeanFactory();

        // Destroy the existing bean instance
        registry.destroySingleton("propertyService");

        // Re-create the bean
        propertyService = (PropertyService) applicationContext.getBean("propertyService");

        // Verify bean has reloaded the new property value
        property1 = propertyService.getProperty1();
        assertThat(property1).isNotNull();
        assertThat(property1).isEqualTo("updated");
        System.out.println("New value of 'property1': " + property1);
    }
}

您可以像这样从项目的根目录运行测试:

mvn test

您应该看到类似于以下内容的输出,表明属性已成功从数据库加载、更新并重新加载到 Spring Beans 中:

Starting Property Source Test
Value of 'property1': initial value
Updating Property 'property1'
Reloading PropertyService Bean
New value of 'property1': updated

从外部存储库动态加载 Spring 属性 |作者:安德斯·斯旺森 | 2024 年 11 月 |中等的 --- Dynamically load Spring properties from external repositories | by Anders Swanson | Nov, 2024 | Medium

 

posted @ 2025-01-10 09:12  CharyGao  阅读(107)  评论(0)    收藏  举报