ZhangZhihuiAAA - 博客园

Spark for ML - Study Notes 4

摘要： multicollinearity import pandas as pd # 示例数据 data = { "X1": [1, 2, 3, 4, 5], "X2": [2, 4, 6, 8, 10], # X2 是 X1 的两倍，完全共线 "X3": [5, 3, 4, 2, 1] } df = p 阅读全文

posted @ 2025-01-30 11:56 ZhangZhihuiAAA 阅读(33) 评论(0) 推荐(0)

VSCode - How to stop terminal from inheriting virtual environments?

摘要：阅读全文

posted @ 2025-01-30 09:38 ZhangZhihuiAAA 阅读(10) 评论(0) 推荐(0)

Spark for ML - Study Notes 3

摘要： from pyspark.sql import SparkSession from pyspark.ml.feature import HashingTF, IDF, Tokenizer spark = SparkSession.builder.appName("TF-IDF Example").g 阅读全文

posted @ 2025-01-28 17:37 ZhangZhihuiAAA 阅读(15) 评论(0) 推荐(0)

Spark for ML - Study Notes 2

摘要： from pyspark.sql import SparkSession spark = SparkSession.builder.appName("HDFS Read Example").getOrCreate() # Define the HDFS path hdfs_path = "hdfs: 阅读全文

posted @ 2025-01-26 22:33 ZhangZhihuiAAA 阅读(17) 评论(0) 推荐(0)

Spark for ML - Study Notes 1

摘要：阅读全文

posted @ 2025-01-26 21:42 ZhangZhihuiAAA 阅读(8) 评论(0) 推荐(0)

Singular Value Decomposition (SVD)

摘要： Singular Value Decomposition (SVD) is a powerful mathematical technique used in linear algebra to factorize a matrix into three simpler matrices. It i 阅读全文

posted @ 2025-01-26 16:37 ZhangZhihuiAAA 阅读(43) 评论(0) 推荐(0)

Spark

摘要： 1. Download: https://spark.apache.org/downloads.html 2. Install: (base) zzh@ZZHPC:~/Downloads/sfw$ tar -xvzf spark-3.5.4-bin-hadoop3.tgz 3. Set enviro 阅读全文

posted @ 2025-01-26 12:30 ZhangZhihuiAAA 阅读(20) 评论(0) 推荐(0)

Airflow - Study Notes 6

摘要： 1. First, we will set up the imports that are required for the dashboard view: from __future__ import annotations from typing import TYPE_CHECKING fro 阅读全文

posted @ 2025-01-20 13:16 ZhangZhihuiAAA 阅读(22) 评论(0) 推荐(0)

Airflow - AirflowTimetableInvalid: Exactly 5, 6 or 7 columns has to be specified for iterator expression.

摘要： from airflow.decorators import ( dag, task, ) from pendulum import datetime @dag( schedule="@none", start_date=datetime(2025, 1, 1), catchup=False, de 阅读全文

posted @ 2025-01-19 23:18 ZhangZhihuiAAA 阅读(167) 评论(0) 推荐(0)

Airflow - Study Notes 5

摘要： To create a new connection, select the + to add a new record. from airflow.decorators import ( dag, task, ) from pendulum import datetime @dag( start_ 阅读全文

posted @ 2025-01-19 22:38 ZhangZhihuiAAA 阅读(5) 评论(0) 推荐(0)

导航

2025年1月30日

2025年1月28日

2025年1月26日

2025年1月20日

2025年1月19日


博客园 © 2004-2025 浙公网安备 33010602011771号浙ICP备2021040463号-3