Scala自定义传参
最近在写Spark导数程序的时候,为了实现程序的多元化,即使用同一套程序,实现不同场景的需求,而参考Spark的org.apache.spark.deploy.master.MasterArguments解析main()方法参数写法
@tailrec
def parameterMatching(args: List[String]): Unit={
val argsList: List[String] = args.toList
argsList match {
case ("--inputPath" | "-i") :: value :: tail =>
hdfsPath = value
parameterMatching(tail)
case ("--execSQL" | "-e") :: value :: tail =>
execSQL = value
parameterMatching(tail)
case ("--outputSchema" | "-s") :: value :: tail =>
outputSchema = value
parameterMatching(tail)
case ("--headLine" | "-h") :: value :: tail =>
headLine = value
parameterMatching(tail)
case ("--writeMode" | "-m") :: value :: tail =>
writeMode = value
parameterMatching(tail)
case ("--outputPath" | "-o") :: value :: tail =>
outputPath = value
parameterMatching(tail)
case ("--numPartitions" | "-n") :: value :: tail =>
numPartitions = value.toInt
parameterMatching(tail)
case ("--help") :: tail =>
printUsageAndExit(0)
case Nil => // No-op
case _ =>
printUsageAndExit(1)
}
}
def printUsageAndExit(exitCode: Int): Unit ={
System.err.println(
"Usage: Hdfs2MppAbsolutePath [options]\n" +
"\n" +
"Options:\n" +
" -i hdfsPath, --inputPath hdfsPath 读HDFS文件路径 (必选。)\n" +
" -e execSQL, --execSQL execSQL 对读数据执行的SQL (可选。default: select * from temp)\n" +
" -s outputSchema, --outputSchema outputSchema 写数据的格式 (可选。default: csv)\n" +
" -h headLine, --headLine headLine 是否打印头标题。(可选。default: true)\n" +
" -m writeMode, --writeMode writeMode 写入数据模式。 (可选。default: append)\n" +
" -o outputPath, --outputPath outputPath 写到本地路径 (必选。)\n" +
" -n numPartitions, --numPartitions numPartitions 设置写入本地的分区数 (可选。default 1)\n"
)
// scalastyle:on println
System.exit(exitCode)
}
届时main()方法内部只要调用这个方法旧可以实现程序更具不同场景,做出不同的导出样式。
def main(args: Array[String]): Unit = {
val argsList: List[String] = args.toList
println("\n 执行参数匹配:")
parameterMatching(argsList)
}
如果有些参数不想传,也可以使用默认值,这样也不会影响该程序的正常使用
object Hdfs2LocalPath {
// 加载变量
private var hdfsPath: String = _
private var execSQL: String = "select * from temp"
private var outputSchema: String = "csv"
private var headLine: String = "true"
private var writeMode: String = "append"
private var outputPath: String = _
private var numPartitions: Int = 1
}

浙公网安备 33010602011771号