ZhangZhihui's Blog  
上一页 1 ··· 23 24 25 26 27 28 29 30 31 ··· 102 下一页

2025年2月3日

摘要: 1. Download Spark 3.4.1 2. Download Java JDK 17 3. Setup Python virtual environment 3.11.9 .bashrc: sfw=~/Downloads/sfw zpy=~/venvs/zpy311 export JAVA 阅读全文
posted @ 2025-02-03 17:33 ZhangZhihuiAAA 阅读(52) 评论(0) 推荐(0)
 
摘要: from delta import configure_spark_with_delta_pip, DeltaTable from pyspark.sql import SparkSession builder = (SparkSession.builder .appName("create-del 阅读全文
posted @ 2025-02-03 12:56 ZhangZhihuiAAA 阅读(25) 评论(0) 推荐(0)

2025年2月2日

摘要: # Apply transform function to Numbers column df_transformed = ( df.select("category", "overallMotivation", "year", "laureates", transform(col("laureat 阅读全文
posted @ 2025-02-02 19:30 ZhangZhihuiAAA 阅读(19) 评论(0) 推荐(0)
 
摘要: build.sh: #!/bin/bash # # -- Build Apache Spark Standalone Cluster Docker Images # # -- Variables # BUILD_DATE="$(date -u +'%Y-%m-%d')" SPARK_VERSION= 阅读全文
posted @ 2025-02-02 15:00 ZhangZhihuiAAA 阅读(22) 评论(0) 推荐(0)

2025年2月1日

摘要: from pyspark.sql.functions import flatten, collect_list # create a DataFrame with an array of arrays column df = spark.createDataFrame([ (1, [[1, 2], 阅读全文
posted @ 2025-02-01 22:45 ZhangZhihuiAAA 阅读(34) 评论(0) 推荐(0)
 
摘要: build.sh: #!/bin/bash # # -- Build Apache Spark Standalone Cluster Docker Images # # -- Variables # BUILD_DATE="$(date -u +'%Y-%m-%d')" SPARK_VERSION= 阅读全文
posted @ 2025-02-01 20:24 ZhangZhihuiAAA 阅读(21) 评论(0) 推荐(0)

2025年1月31日

摘要: 阅读全文
posted @ 2025-01-31 11:27 ZhangZhihuiAAA 阅读(8) 评论(0) 推荐(0)

2025年1月30日

摘要: multicollinearity import pandas as pd # 示例数据 data = { "X1": [1, 2, 3, 4, 5], "X2": [2, 4, 6, 8, 10], # X2 是 X1 的两倍,完全共线 "X3": [5, 3, 4, 2, 1] } df = p 阅读全文
posted @ 2025-01-30 11:56 ZhangZhihuiAAA 阅读(33) 评论(0) 推荐(0)
 
摘要: 阅读全文
posted @ 2025-01-30 09:38 ZhangZhihuiAAA 阅读(10) 评论(0) 推荐(0)

2025年1月28日

摘要: from pyspark.sql import SparkSession from pyspark.ml.feature import HashingTF, IDF, Tokenizer spark = SparkSession.builder.appName("TF-IDF Example").g 阅读全文
posted @ 2025-01-28 17:37 ZhangZhihuiAAA 阅读(15) 评论(0) 推荐(0)
上一页 1 ··· 23 24 25 26 27 28 29 30 31 ··· 102 下一页