JDBC注册原理与自定义类加载器解决com.cloudera.hive.jdbc41.HS2Driver的加载【重点】
问题：maven systemPath的jar不被maven assembly插件所打包，导致运行期找不到相应的driver类；而且必须打成一个jar包，不能用maven-jar-plugin + lib形式
解决方案：
将driver HiveJdbc41.jar放于resource，作为普通资源，运行期使用自定义类加载器加载 （使用resource中的jar包资源作为UrlClassloader（二））
为了方便，本文使用mysql的jdbc做测试
0
https://yanbin.blog/custom-classload-dynamic-load-jdbc-driver/#more-8187 http://www.kfu.com/~nsayer/Java/dyn-jdbc.html https://blog.csdn.net/d6619309/article/details/53149384 https://blog.csdn.net/qq_35385196/article/details/81750639 http://www.hackerav.com/?post=44723
上面5个链接试图解决动态使用自定义类加载器加载mysql driver，但我尝试都失败了，我自己想办法
1 通过 https://blog.csdn.net/yangcheng33/article/details/52631940 研究jdbc的类加载器check机制
@CallerSensitive
public static Connection getConnection(String url,
String user, String password) throws SQLException {
java.util.Properties info = new java.util.Properties();
if (user != null) {
info.put("user", user);
}
if (password != null) {
info.put("password", password);
}
return (getConnection(url, info, Reflection.getCallerClass()));
}
private static Connection getConnection(
String url, java.util.Properties info, Class<?> caller) throws SQLException {
/*
* When callerCl is null, we should check the application's
* (which is invoking this class indirectly)
* classloader, so that the JDBC driver class outside rt.jar
* can be loaded from here.
*/
ClassLoader callerCL = caller != null ? caller.getClassLoader() : null;
synchronized(DriverManager.class) {
// synchronize loading of the correct classloader.
if (callerCL == null) {
callerCL = Thread.currentThread().getContextClassLoader();
}
}
if(url == null) {
throw new SQLException("The url cannot be null", "08001");
}
println("DriverManager.getConnection(\"" + url + "\")");
// Walk through the loaded registeredDrivers attempting to make a connection.
// Remember the first exception that gets raised so we can reraise it.
SQLException reason = null;
for(DriverInfo aDriver : registeredDrivers) {
// If the caller does not have permission to load the driver then
// skip it.
if(isDriverAllowed(aDriver.driver, callerCL)) {
try {
println(" trying " + aDriver.driver.getClass().getName());
Connection con = aDriver.driver.connect(url, info);
if (con != null) {
// Success!
println("getConnection returning " + aDriver.driver.getClass().getName());
return (con);
}
} catch (SQLException ex) {
if (reason == null) {
reason = ex;
}
}
} else {
println(" skipping: " + aDriver.getClass().getName());
}
}
// if we got here nobody could connect.
if (reason != null) {
println("getConnection failed: " + reason);
throw reason;
}
println("getConnection: no suitable driver found for "+ url);
throw new SQLException("No suitable driver found for "+ url, "08001");
}
private static boolean isDriverAllowed(Driver driver, ClassLoader classLoader) {
boolean result = false;
if(driver != null) {
Class<?> aClass = null;
try {
aClass = Class.forName(driver.getClass().getName(), true, classLoader); driver由自定义类加载器加载，而传入的是系统类加载器
} catch (Exception ex) {
result = false;
}
result = ( aClass == driver.getClass() ) ? true : false;
}
return result;
}
2 根据1中的分析，问题的关键在于这个：
mysql driver所在类加载器（自定义动态类加载器）与getconnection调用方类的类加载器必须是同一个，故需要一个代理人，突破DriverManger中Reflection.getCallerClass()的防线
这个代理类必须是系统类加载器不可见，而且这个代理类要由加载driver的自定义类加载器来加载，
具体有n种方式：
2.1 使用手动编译代理，放入resource；
缺点：部署麻烦，要先编译代理人，然后将代理人放入resource，然后再编译主项目，两次编译不符合公司要求
2.2 java文件作为文本资源，使用运行期编译；
缺点是运行期编译在各生产环境不可控；https://www.cnblogs.com/chenyf/p/10246154.html
2.3 先手写一个类proxyA，运行期获取proxyA的字节码，
2.3.1 使用asm或javassist copy一个proxyB，并重命名，搞在自定义类加载器的findclass中；
2.3.2 或不重命名，主程序强行findClass绕过系统类加载器中的同名proxyA
2.3.3 loadclass/findclass一个特定的类名，findclass中碰到这个就去copy A，变相偷着实现2.3.1——最终采纳
这样系统类加载器加载A，自定义加载器加载B，反射调用B去getconnection，drivermanager检查的时候仍然会发现driver加载器和调用类加载器都是自定义类加载器包bingo
2.4 使用asm或其他方式（java agent 加载器织入——java.lang.instrument包 AOP，使用javassist ，jdk动态代理源码底层（jdk生成字节码及5种字节码生产方式））造一个类；
太流氓，成本大，风险大，需要对字节码结构很了解
最终采用2.3.3
3 代码
src/main/java/lc3
jars
JdbcProxy.java
mysql-connector-java-=5.1.39.jar
Query
MyJdbcLoader
3.1 解压式（tomcat）
package lc3;
import lc3.jars.Query;
import java.io.*;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.sql.*;
import static lc3.jars.Query.URL;
import static lc3.jars.Query.PASSWORD;
import static lc3.jars.Query.USER;
/**
* https://www.cnblogs.com/silyvin/articles/12166228.html
* 2.3.3 3.1
* Created by joyce on 2020/1/9.
*/
public class MyJdbcLoader_3_1 {
public static void main(String [] f) throws Exception {
try {
/**
* 这个在打包后没用，URLClassLoader无法加载jar中jar
* ide可以是因为会将jar释放到target目录
*/
// URL url = MyJdbcLoader.class.getResource("jars/mysql-connector-java-5.1.39.jar");
InputStream inputStream = MyJdbcLoader_3_1.class.getResourceAsStream("jars/mysql-connector-java-5.1.39.jar");
URL url = copyJar(inputStream, "tmp-mysql-connector-java-5.1.39.jar");
MyClassLoader classLoader = new MyClassLoader(new URL[]{url});
String driverClass = "com.mysql.jdbc.Driver";
Class.forName(driverClass, true, classLoader);
/*
loaclass 即可，不需要强制findclass，因为父系统类加载器只有lc3.jars.JdbcProxy
*/
Class proxy = classLoader.loadClass("MyJdbcProxy");
// Class proxy = classLoader.findClass("JdbcProxy");
Method method = proxy.getMethod("getConnection", String.class, String.class, String.class);
Connection connection = (Connection)method.invoke(null, URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
private static class MyClassLoader extends URLClassLoader {
public MyClassLoader(java.net.URL[] urls) {
super(urls);
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
if(!name.equals("MyJdbcProxy"))
return super.findClass(name);
try {
InputStream inputStream = this.getParent().getResourceAsStream("lc3/jars/JdbcProxy.class");
byte [] tmp = new byte[1024];
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int len = -1;
int lenTotal = 0;
while ((len = inputStream.read(tmp)) != -1) {
lenTotal += len;
byteArrayOutputStream.write(tmp, 0, len);
}
byte [] bytes = byteArrayOutputStream.toByteArray();
if(bytes.length != lenTotal)
throw new RuntimeException("copy JdbcProxy error");
return defineClass(bytes, 0, bytes.length);
} catch (Exception e) {
e.printStackTrace();
}
return super.findClass(name);
}
}
private static URL copyJar(InputStream inputStream, String name) throws IOException {
File exist = new File(name);
if(exist.exists())
return new URL("file:" + name);
int len = -1;
byte [] bytes = new byte[1024];
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
while ((len = inputStream.read(bytes)) != -1) {
byteArrayOutputStream.write(bytes, 0, len);
}
inputStream.close();
File file = new File(name);
OutputStream outputStream = new FileOutputStream(file);
outputStream.write(byteArrayOutputStream.toByteArray());
outputStream.close();
URL url = new URL("file:" + name);
return url;
}
}
3.2
package lc3;
import lc3.jars.Query;
import java.io.*;
import java.lang.reflect.Method;
import java.net.URL;
import java.net.URLClassLoader;
import java.sql.Connection;
import java.sql.Driver;
import java.sql.DriverManager;
import java.util.HashMap;
import java.util.jar.JarEntry;
import java.util.jar.JarInputStream;
import java.util.zip.ZipEntry;
import static lc3.jars.Query.*;
/**
* https://www.cnblogs.com/silyvin/articles/12166228.html
* 2.3.3 3.1
* Created by joyce on 2020/1/9.
*/
public class MyJdbcLoader_3_2 {
public static void main(String [] f) throws Exception {
try {
/**
* 5.1.39不行
*/
// InputStream inputStream = MyJdbcLoader_3_2.class.getResourceAsStream("jars/mysql-connector-java-5.1.39.jar");
// InputStream inputStream = MyJdbcLoader_3_2.class.getResourceAsStream("jars/mysql-connector-java-5.1.39-bin.jar");
InputStream inputStream = MyJdbcLoader_3_2.class.getResourceAsStream("jars/mysql-connector-java-5.1.0-bin.jar");
MyClassLoader classLoader = new MyClassLoader(new JarInputStream[]{new JarInputStream(inputStream)});
String driverClass = "com.mysql.jdbc.Driver";
Class cl = Class.forName(driverClass, true, classLoader);
Class proxy = classLoader.loadClass("MyJdbcProxy");
Method method = proxy.getMethod("getConnection", String.class, String.class, String.class);
Connection connection = (Connection)method.invoke(null, URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
private static class MyClassLoader extends ClassLoader {
JarInputStream [] list = null;
private HashMap<String, byte[]> classes = new HashMap<>();
public MyClassLoader(JarInputStream [] jarInputStream) {
this.list = jarInputStream;
for(JarInputStream jar : list) {
JarEntry entry;
try {
while ((entry = jar.getNextJarEntry()) != null) {
String name = entry.getName();
ByteArrayOutputStream out = new ByteArrayOutputStream();
int len = -1;
byte [] tmp = new byte[1024];
while ((len = jar.read(tmp)) != -1) {
out.write(tmp, 0, len);
}
byte[] bytes = out.toByteArray();
classes.put(name, bytes);
}
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("total classes - " + classes.size());
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
if(!name.equals("MyJdbcProxy"))
try {
InputStream in = getResourceAsStream(name.replace('.', '/') + ".class");
ByteArrayOutputStream out = new ByteArrayOutputStream();
int len = -1;
byte [] tmp = new byte[1024];
while ((len = in.read(tmp)) != -1) {
out.write(tmp, 0, len);
}
byte[] bytes = out.toByteArray();
return defineClass(name, bytes, 0, bytes.length);
} catch (Exception e) {
e.printStackTrace();
}
try {
InputStream inputStream = this.getParent().getResourceAsStream("lc3/jars/JdbcProxy.class");
byte [] tmp = new byte[1024];
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int len = -1;
int lenTotal = 0;
while ((len = inputStream.read(tmp)) != -1) {
lenTotal += len;
byteArrayOutputStream.write(tmp, 0, len);
}
byte [] bytes = byteArrayOutputStream.toByteArray();
if(bytes.length != lenTotal)
throw new RuntimeException("copy JdbcProxy error");
return defineClass(bytes, 0, bytes.length);
} catch (Exception e) {
e.printStackTrace();
}
return super.findClass(name);
}
@Override
public InputStream getResourceAsStream(String name) {
System.out.println("getResourceAsStream - " + name);
if(classes.containsKey(name)) {
return new ByteArrayInputStream(classes.get(name));
}
System.out.println("getResourceAsStream - error - " + name);
return super.getResourceAsStream(name);
}
}
}
5.1.39 mysql、mysql-bin不成功，5.1.0-bin成功，此法不稳定可见一斑
期间报错：
total classes - 207 getResourceAsStream - com/mysql/jdbc/Driver.class getResourceAsStream - com/mysql/jdbc/NonRegisteringDriver.class driver:com.mysql.jdbc.Driver@2b193f2d:lc3.MyJdbcLoader_3_2$MyClassLoader@266474c2 getResourceAsStream - com/mysql/jdbc/StringUtils.class getResourceAsStream - com/mysql/jdbc/Connection.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties.class getResourceAsStream - com/mysql/jdbc/NotImplemented.class getResourceAsStream - com/mysql/jdbc/PreparedStatement.class getResourceAsStream - com/mysql/jdbc/Statement.class getResourceAsStream - com/mysql/jdbc/ServerPreparedStatement.class getResourceAsStream - com/mysql/jdbc/ResultSet.class getResourceAsStream - com/mysql/jdbc/util/LRUCache.class getResourceAsStream - com/mysql/jdbc/Connection$1.class getResourceAsStream - com/mysql/jdbc/log/Log.class getResourceAsStream - com/mysql/jdbc/log/StandardLogger.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties$BooleanConnectionProperty.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties$ConnectionProperty.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties$MemorySizeConnectionProperty.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties$IntegerConnectionProperty.class getResourceAsStream - com/mysql/jdbc/ConnectionProperties$StringConnectionProperty.class getResourceAsStream - com/mysql/jdbc/log/NullLogger.class getResourceAsStream - com/mysql/jdbc/Constants.class getResourceAsStream - com/mysql/jdbc/Util.class getResourceAsStream - com/mysql/jdbc/JDBC4Connection.class getResourceAsStream - com/mysql/jdbc/exceptions/NotYetImplementedException.class getResourceAsStream - com/mysql/jdbc/JDBC4Connection$1.class getResourceAsStream - com/mysql/jdbc/StandardSocketFactory.class getResourceAsStream - com/mysql/jdbc/SocketFactory.class getResourceAsStream - com/mysql/jdbc/CharsetMapping.class getResourceAsStream - com/mysql/jdbc/VersionedStringProperty.class getResourceAsStream - com/mysql/jdbc/log/LogFactory.class getResourceAsStream - com/mysql/jdbc/MysqlIO.class getResourceAsStream - com/mysql/jdbc/RowData.class getResourceAsStream - com/mysql/jdbc/ConnectionFeatureNotAvailableException.class getResourceAsStream - com/mysql/jdbc/CommunicationsException.class getResourceAsStream - com/mysql/jdbc/StreamingNotifiable.class getResourceAsStream - com/mysql/jdbc/PacketTooBigException.class getResourceAsStream - com/mysql/jdbc/Buffer.class getResourceAsStream - com/mysql/jdbc/MysqlDataTruncation.class getResourceAsStream - com/mysql/jdbc/CompressedInputStream.class getResourceAsStream - com/mysql/jdbc/util/ReadAheadInputStream.class getResourceAsStream - com/mysql/jdbc/Security.class getResourceAsStream - com/mysql/jdbc/Statement$CancelTask.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLTimeoutException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLTransientException.class getResourceAsStream - com/mysql/jdbc/Field.class getResourceAsStream - com/mysql/jdbc/MysqlDefs.class getResourceAsStream - com/mysql/jdbc/RowDataStatic.class getResourceAsStream - com/mysql/jdbc/NotUpdatable.class getResourceAsStream - com/mysql/jdbc/UpdatableResultSet.class getResourceAsStream - com/mysql/jdbc/JDBC4ResultSet.class getResourceAsStream - com/mysql/jdbc/JDBC4UpdatableResultSet.class getResourceAsStream - com/mysql/jdbc/SingleByteCharsetConverter.class getResourceAsStream - com/mysql/jdbc/EscapeProcessor.class getResourceAsStream - com/mysql/jdbc/LicenseConfiguration.class getResourceAsStream - com/mysql/jdbc/DatabaseMetaData.class getResourceAsStream - com/mysql/jdbc/DatabaseMetaData$SingleStringIterator.class getResourceAsStream - com/mysql/jdbc/DatabaseMetaData$IteratorWithCleanup.class getResourceAsStream - com/mysql/jdbc/DatabaseMetaData$ResultSetIterator.class getResourceAsStream - com/mysql/jdbc/DatabaseMetaDataUsingInfoSchema.class getResourceAsStream - com/mysql/jdbc/JDBC4DatabaseMetaData.class getResourceAsStream - com/mysql/jdbc/JDBC4DatabaseMetaDataUsingInfoSchema.class getResourceAsStream - com/mysql/jdbc/JDBC4PreparedStatement.class getResourceAsStream - com/mysql/jdbc/PreparedStatement$ParseInfo.class getResourceAsStream - com/mysql/jdbc/SQLError.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLTransientConnectionException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLNonTransientConnectionException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLNonTransientException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLDataException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLIntegrityConstraintViolationException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLSyntaxErrorException.class getResourceAsStream - com/mysql/jdbc/exceptions/MySQLTransactionRollbackException.class getResourceAsStream - com/mysql/jdbc/exceptions/jdbc4/CommunicationsException.class getResourceAsStream - com/mysql/jdbc/Messages.class getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages.class getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages.class java.lang.NullPointerException at lc3.MyJdbcLoader_3_2$MyClassLoader.findClass(MyJdbcLoader_3_2.java:86) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at com.mysql.jdbc.Messages.<clinit>(Messages.java:54) at com.mysql.jdbc.SQLError.<clinit>(SQLError.java:178) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2918) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1601) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1710) at com.mysql.jdbc.Connection.execSQL(Connection.java:2436) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1402) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1556) at lc3.jars.Query.query(Query.java:26) at lc3.MyJdbcLoader_3_2.main(MyJdbcLoader_3_2.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages.properties java.lang.NullPointerException at lc3.MyJdbcLoader_3_2$MyClassLoader.findClass(MyJdbcLoader_3_2.java:86) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at com.mysql.jdbc.Messages.<clinit>(Messages.java:54) at com.mysql.jdbc.SQLError.<clinit>(SQLError.java:178) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2918) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1601) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1710) at com.mysql.jdbc.Connection.execSQL(Connection.java:2436) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1402) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1556) at lc3.jars.Query.query(Query.java:26) at lc3.MyJdbcLoader_3_2.main(MyJdbcLoader_3_2.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh.class getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh.class getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh.properties getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh.properties getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh_CN.class getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh_CN.class java.lang.NullPointerException at lc3.MyJdbcLoader_3_2$MyClassLoader.findClass(MyJdbcLoader_3_2.java:86) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at com.mysql.jdbc.Messages.<clinit>(Messages.java:54) at com.mysql.jdbc.SQLError.<clinit>(SQLError.java:178) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2918) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1601) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1710) at com.mysql.jdbc.Connection.execSQL(Connection.java:2436) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1402) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1556) at lc3.jars.Query.query(Query.java:26) at lc3.MyJdbcLoader_3_2.main(MyJdbcLoader_3_2.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) java.lang.NullPointerException at lc3.MyJdbcLoader_3_2$MyClassLoader.findClass(MyJdbcLoader_3_2.java:86) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640) at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465) at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361) at java.util.ResourceBundle.getBundle(ResourceBundle.java:1082) at com.mysql.jdbc.Messages.<clinit>(Messages.java:54) at com.mysql.jdbc.SQLError.<clinit>(SQLError.java:178) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2918) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1601) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1710) at com.mysql.jdbc.Connection.execSQL(Connection.java:2436) at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1402) at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1556) at lc3.jars.Query.query(Query.java:26) at lc3.MyJdbcLoader_3_2.main(MyJdbcLoader_3_2.java:40) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) java.lang.NullPointerException at lc3.MyJdbcLoader_3_2$MyClassLoader.findClass(MyJdbcLoader_3_2.java:86) getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh_CN.properties getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh_CN.properties getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans.class getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans.class getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans.properties getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans.properties getResourceAsStream - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans_CN.class getResourceAsStream - error - com/mysql/jdbc/LocalizedErrorMessages_zh_Hans_CN.class
3.3
package lc3;
import lc3.jars.Query;
import java.io.*;
import java.lang.reflect.Method;
import java.net.*;
import java.sql.Connection;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import static lc3.jars.Query.*;
/**
* https://www.cnblogs.com/silyvin/articles/12166228.html
* 2.3.3 3.3
* Created by joyce on 2020/1/9.
*/
public class MyJdbcLoader_3_3 {
public static void main(String [] f) throws Exception {
try {
List<URL> urls = MyClassLoader.init(new String []{"jars/mysql-connector-java-5.1.39.jar"});
MyClassLoader classLoader = new MyClassLoader(urls.toArray(new java.net.URL [urls.size()]));
String driverClass = "com.mysql.jdbc.Driver";
Class.forName(driverClass, true, classLoader);
Class proxy = classLoader.loadClass("MyJdbcProxy");
Method method = proxy.getMethod("getConnection", String.class, String.class, String.class);
Connection connection = (Connection)method.invoke(null, URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
private static class MyClassLoader extends URLClassLoader {
public static List<URL> init(String [] resourceJars) throws Exception {
List<java.net.URL> urls = new ArrayList<>();
Map<String, ByteArrayOutputStream> map = new ConcurrentHashMap<>();
java.net.URL.setURLStreamHandlerFactory(new URLStreamHandlerFactory() {
public URLStreamHandler createURLStreamHandler(String urlProtocol) {
System.out.println("Someone asked for protocol: " + urlProtocol);
if ("myjarprotocol".equalsIgnoreCase(urlProtocol)) {
return new URLStreamHandler() {
@Override
protected URLConnection openConnection(URL url) throws IOException {
String key = url.toString().split(":")[1];
return new URLConnection(url) {
public void connect() throws IOException {}
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(map.get(key).toByteArray());
}
};
}
};
}
return null;
}
});
for(String resourceJar : resourceJars) {
InputStream in = MyClassLoader.class.getResourceAsStream(resourceJar);
int len = -1;
byte [] bytes = new byte[1024];
ByteArrayOutputStream jarBytes = new ByteArrayOutputStream();
while ((len = in.read(bytes)) != -1) {
jarBytes.write(bytes, 0, len);
}
map.put(resourceJar, jarBytes);
urls.add(new URL("myjarprotocol:" + resourceJar));
}
return urls;
}
public MyClassLoader(java.net.URL[] urls) {
super(urls);
}
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
if(!name.equals("MyJdbcProxy"))
return super.findClass(name);
try {
InputStream inputStream = this.getParent().getResourceAsStream("lc3/jars/JdbcProxy.class");
byte [] tmp = new byte[1024];
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int len = -1;
int lenTotal = 0;
while ((len = inputStream.read(tmp)) != -1) {
lenTotal += len;
byteArrayOutputStream.write(tmp, 0, len);
}
byte [] bytes = byteArrayOutputStream.toByteArray();
if(bytes.length != lenTotal)
throw new RuntimeException("copy JdbcProxy error");
return defineClass(bytes, 0, bytes.length);
} catch (Exception e) {
e.printStackTrace();
}
return super.findClass(name);
}
}
}
3.4 其实系统类加载器也是可以添加URL的，不需要自定义类加载器，这样的话，2中的问题就不存在了
当然，这样的话，mysql-connector-5.1.39.jar这个包进入系统类加载器，就有可能导致冲突
package lc3;
import lc3.jars.JdbcProxy;
import lc3.jars.Query;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Method;
import java.net.*;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import static lc3.jars.Query.*;
/**
* https://www.cnblogs.com/silyvin/articles/12166228.html
* 3.4
* Created by joyce on 2020/1/9.
*/
public class MyJdbcLoader_3_4 {
public static void main(String [] f) throws Exception {
try {
List<URL> list = init(new String [] {"jars/mysql-connector-java-5.1.39.jar"});
URLClassLoader systemClassloader = (URLClassLoader) ClassLoader.getSystemClassLoader();
Method systemClassloaderMethod = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
systemClassloaderMethod.setAccessible(true);
for(URL url : list) {
systemClassloaderMethod.invoke(systemClassloader, url);
}
/**
* 此处直接用系统类加载器的JdbcProxy
*/
Connection connection = (Connection) JdbcProxy.getConnection(URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
public static List<URL> init(String [] resourceJars) throws Exception {
List<java.net.URL> urls = new ArrayList<>();
Map<String, ByteArrayOutputStream> map = new ConcurrentHashMap<>();
java.net.URL.setURLStreamHandlerFactory(new URLStreamHandlerFactory() {
public URLStreamHandler createURLStreamHandler(String urlProtocol) {
System.out.println("Someone asked for protocol: " + urlProtocol);
if ("myjarprotocol".equalsIgnoreCase(urlProtocol)) {
return new URLStreamHandler() {
@Override
protected URLConnection openConnection(URL url) throws IOException {
String key = url.toString().split(":")[1];
return new URLConnection(url) {
public void connect() throws IOException {}
public InputStream getInputStream() throws IOException {
return new ByteArrayInputStream(map.get(key).toByteArray());
}
};
}
};
}
return null;
}
});
for(String resourceJar : resourceJars) {
InputStream in = MyJdbcLoader_3_4.class.getResourceAsStream(resourceJar);
int len = -1;
byte [] bytes = new byte[1024];
ByteArrayOutputStream jarBytes = new ByteArrayOutputStream();
while ((len = in.read(bytes)) != -1) {
jarBytes.write(bytes, 0, len);
}
map.put(resourceJar, jarBytes);
urls.add(new URL("myjarprotocol:" + resourceJar));
}
return urls;
}
}
4 最终采用3.3+2.3.3的方式
原因：
4.1 类隔离，防止冲突，侵入性低，所以不选用3.4方式
4.2 通过伪造URL的方式，仍然可以直接使用JDK的URLClassLoader，够问题，比自己实现一个ClassLoader稳定，比如本文的3.2，就表现出了不稳定
4.3 3.1的方式要将jar写入磁盘，东西都进内存了，还写进磁盘，太多此一举，太low
2021.4.30
第4种打整包插件，urlfactory already set 补充了方式3.3的劣势，以及我们在tomcat项目里面最终没有使用的原因
5 出现了defineClass重复定义错误，我们以3.3为例（hive+javaserver时使用3.3，spring+sybase时切到3.2，因为URL注册与spring boot冲突）：
public static void main(String [] f) throws Exception {
try {
List<URL> urls = MyClassLoader.init(new String []{"jars/mysql-connector-java-5.1.39.jar"});
MyClassLoader classLoader = new MyClassLoader(urls.toArray(new java.net.URL [urls.size()]));
String driverClass = "com.mysql.jdbc.Driver";
Class.forName(driverClass, true, classLoader);
Class proxy = classLoader.loadClass("MyJdbcProxy");
classLoader.loadClass("MyJdbcProxy"); 增加
Method method = proxy.getMethod("getConnection", String.class, String.class, String.class);
Connection connection = (Connection)method.invoke(null, URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
报错：
Exception in thread "main" java.lang.LinkageError: loader (instance of lc3/MyJdbcLoader_3_3$MyClassLoader): attempted duplicate class definition for name: "lc3/jars/JdbcProxy"
我们试着跟踪一下代码：
第一次loadclass MyJdbcProxy
MyClassLoader查看缓存（Class<?> c = findLoadedClass(name);）有无名为MyJdbcProxy的类——没有
MyClassLoader交给父加载器，因为MyJdbcProxy父加载器也没有，因为我们改名了lc3.jars.JdbcProxy---->MyJdbcProxy——也没有
MyClassLoader调用findClass，然后defineClass，native方法将类缓存入MyClassLoader
MyClassLoader第二次loadClass MyJdbcProxy，再次跑到了defineClass，导致异常
这里就有一个问题：
为什么第二次load在缓存中没找到？
问题的关键出在defineClass，看错误日志，重复定义的是lc3.jar.JdbcProxy，不是MyJdbcProxy，那么defineClass中定义的字节数组，没有经过ASM、javassist、或jdk修改，定义进去得仍然是lc3.jar.JdbcProxy，往上面第5行加粗的缓存进MyClassLoder的也是这个类名而不是MyJdbcProxy，这就解释了为什么第二次load没有找到缓存的字节码，而重新又去findClass、defineClass
那么我们能不能defineClass时指定MyJdbcProxy为类名？答案是不行的，因为字节码没修改过
这个情况与 使用resource中的jar包资源作为UrlClassloader（二） 中 2 不同，该文中，findClass只会调用一次（loadClass的name和defineClass字节码中的类名相同），而在这个案例中，对MyJdbcProxy的loadClass将每次都调用findClass，我们需要自己做缓存
问题找到了，解决方案为，第二次findClass开始，不重新defineClass，将第一次defineClass的返回Class对象缓存起来，第二次直接返回：
package lc3;
public class MyJdbcLoader_3_3 {
public static void main(String [] f) throws Exception {
try {
List<URL> urls = MyClassLoader.init(new String []{"jars/mysql-connector-java-5.1.39.jar"});
MyClassLoader classLoader = new MyClassLoader(urls.toArray(new java.net.URL [urls.size()]));
String driverClass = "com.mysql.jdbc.Driver";
Class.forName(driverClass, true, classLoader);
Class proxy = classLoader.loadClass("MyJdbcProxy");
/**
* 此处原先将导致attempted duplicate class definition for name: "lc3/jars/JdbcProxy"
* 出于偷懒，只有3.3修改了代码，增加了缓存
*/
classLoader.loadClass("MyJdbcProxy");
Method method = proxy.getMethod("getConnection", String.class, String.class, String.class);
Connection connection = (Connection)method.invoke(null, URL, USER, PASSWORD);
Query.query(connection);
} catch (Exception e) {
e.printStackTrace();
}
}
private static class MyClassLoader extends URLClassLoader {
private volatile Class aClass = null;
..........
@Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
if(!name.equals("MyJdbcProxy"))
return super.findClass(name);
if(name.equals("MyJdbcProxy") && aClass != null)
return aClass;
synchronized (MyClassLoader.class) {
if(aClass != null)
return aClass;
try {
InputStream inputStream = this.getParent().getResourceAsStream("lc3/jars/JdbcProxy.class");
.........
aClass = defineClass(bytes, 0, bytes.length);
return aClass;
} catch (Exception e) {
e.printStackTrace();
}
}
return super.findClass(name);
}
}
}
借用单例模式的volatile+doublecheck搞定多线程环境下的loadClass，虽然可能不必要
在此，我们回顾下：自定义类加载器 与 热加载中的情况，
public static void main(String[] args) throws MalformedURLException, ClassNotFoundException, IllegalAccessException, InstantiationException {
// System.out.println(System.getProperty("java.class.path"));
// System.out.println(System.getProperty("user.dir"));
String dir = "file:/Users/sunyuming/Documents/tool/jars//MySub-1.0.0-jar-with-dependencies.jar";
URL url = new URL(dir);
URL[] urls2 = {url};
// 若不指定parent参数，则默认由系统类加载器担任自定义类加载器的父加载器，输出parent:sun.misc.Launcher$AppClassLoader@3764951d
MyUrlClassLoader myUrlClassLoader = new MyUrlClassLoader(urls2);
System.out.println("parent:--" + myUrlClassLoader.getParent());
// 由于A在父加载器（系统类加载器）classpath下，根据双亲委派，优先由父加载器加载，输出A:sun.misc.Launcher$AppClassLoader@3764951d
Class CA = myUrlClassLoader.loadClass("lc.A");
System.out.println("A:--" + CA.getClassLoader());
CA.newInstance();
// 打破双亲委派机制，直接使用findclass绕开从父加载器寻找并由父加载器加载这一步，输出A:lc.Main$MyUrlClassLoader@5acf9800
// 当类加载请求到来时，先从缓存中查找该类对象，如果存在直接返回，如果不存在则交给该类加载去的父加载器去加载，倘若没有父加载则交给顶级启动类加载器去加载，最后倘若仍没有找到，则使用findClass()方法去加载
CA = myUrlClassLoader.findClass("lc.A");
/**
* https://www.cnblogs.com/silyvin/articles/12166228.html
* 此处会导致attempted duplicate class definition for name: "lc/A"
*/
// CA = myUrlClassLoader.findClass("lc.A");
System.out.println("A:--" + CA.getClassLoader());
// 实例化A，此时触发A的static代码块和B的加载，输出A:lc.Main$MyUrlClassLoader@5acf9800
// 由于B在父加载器classpath下，优先由系统类加载器加载，输出B:sun.misc.Launcher$AppClassLoader@3764951d
CA.newInstance();
}
6 遵照使用resource中的jar包资源作为UrlClassloader（二）对3.2、3.3、3.4的方式堆中的map在defineClass及return URL后remove，节约堆内存
浙公网安备 33010602011771号