subtract
Return an RDD with the elements from `this` that are not in `other` .
def subtract(other: RDD[T]): RDD[T]
def subtract(other: RDD[T], numPartitions: Int): RDD[T]
def subtract(other: RDD[T], p: Partitioner): RDD[T]
val a = sc.parallelize(1 to 5) val b = sc.parallelize(1 to 3) val c = a.subtract(b) c.collect Array[Int] = Array(4, 5)
intersection
Return the intersection of this RDD and another one. The output will not contain any duplicate elements, even if the input RDDs did. 交集
def intersection(other: RDD[T], numPartitions: Int): RDD[T] def intersection(other: RDD[T], partitioner: Partitioner)(implicit ord: Ordering[T] = null): RDD[T] def intersection(other: RDD[T]): RDD[T]
val x = sc.parallelize(1 to 10) val y = sc.parallelize(2 to 8) val z = x.intersection(y) z.collect Array[Int] = Array(4, 6, 8, 2, 3, 7, 5)
cartesian
Return the Cartesian product of this RDD and another one, that is, the RDD of all pairs of elements (a, b) where a is in `this` and b is in `other` . 笛卡尔积
def cartesian[U: ClassTag](other: RDD[U]): RDD[(T, U)]
val x = sc.parallelize(List(1,2,3)) val y = sc.parallelize(List(6,7,8)) x.cartesian(y).collect Array[(Int, Int)] = Array((1,6), (1,7), (1,8), (2,6), (3,6), (2,7), (2,8), (3,7), (3,8))
· 「EF Core」框架是如何识别实体类的属性和主键的
· 独立开发,这条路可行吗?
· 我在厂里搞 wine 的日子
· 如何通过向量化技术比较两段文本是否相似?
· 35+程序员的转型之路:经济寒冬中的希望与策略
· 推荐 5 款实用的 Docker 可视化管理工具,工作效率翻倍!
· 瞧瞧别人家的接口重试,那叫一个优雅!
· 独立开发第二周:构建、执行、规划
· 【EF Core】框架是如何识别实体类的属性和主键的
· 简单记录下最近2个月完成的线上系统迁移工作