papering - 博客园

2017年11月2日

摘要： HDFS 压缩性能最优、扫描速度最快；不支持随机访问，仅支持昂贵、复杂的文件查询 HBase适合随机访问 Solr 适合检索需求 HBase访问单个记录的时间为毫秒级别，而HDFS不支持随机访问。 HBase的扫描速度大约是HDFS的1/10-1/8。 HBase 深入浅出 https://www. 阅读全文

posted @ 2017-11-02 09:08 papering 阅读(216) 评论(0) 推荐(0)

graph querying

摘要： querying 阅读全文

posted @ 2017-11-02 00:12 papering 阅读(153) 评论(0) 推荐(0)

2017年11月1日

Java代码优化（长期更新）

摘要： Java代码优化（长期更新）阅读全文

posted @ 2017-11-01 23:53 papering 阅读(163) 评论(0) 推荐(0)

A Short Introduction to Boosting

摘要： http://www.site.uottawa.ca/~stan/csi5387/boost-tut-ppr.pdf Boosting is a general method for improving the accuracy of any given learning algorithm. Th 阅读全文

posted @ 2017-11-01 23:32 papering 阅读(392) 评论(0) 推荐(0)

Boosting AdaBoosting Algorithm

摘要： http://math.mit.edu/~rothvoss/18.304.3PM/Presentations/1-Eric-Boosting304FinalRpdf.pdf Consider MIT Admissions 【qualitative quantitative】 • 2-class sy 阅读全文

posted @ 2017-11-01 21:46 papering 阅读(369) 评论(0) 推荐(0)

Transforming Auto-encoders

摘要： http://www.cs.toronto.edu/~hinton/absps/transauto6.pdf The artificial neural networks that are used to recognize shapes typically use one or more laye 阅读全文

posted @ 2017-11-01 16:58 papering 阅读(482) 评论(0) 推荐(0)

Dynamic Routing Between Capsules

摘要： https://arxiv.org/abs/1710.09829 阅读全文

posted @ 2017-11-01 16:49 papering 阅读(351) 评论(0) 推荐(0)

empirical risk minimization

摘要： Empirical_risk_minimization 阅读全文

posted @ 2017-11-01 16:28 papering 阅读(193) 评论(0) 推荐(0)

Reducing the Dimensionality of Data with Neural Networks

摘要： http://www.cs.toronto.edu/~hinton/science.pdf High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network 阅读全文

posted @ 2017-11-01 16:08 papering 阅读(452) 评论(0) 推荐(0)

Geoffrey E. Hinton

摘要： https://www.cs.toronto.edu/~hinton/ Geoffrey E. Hinton I am an Engineering Fellow at Google where I manage Brain Team Toronto, which is a new part of 阅读全文

posted @ 2017-11-01 15:56 papering 阅读(774) 评论(0) 推荐(0)

Dropout: A Simple Way to Prevent Neural Networks fromOverfitting

摘要： https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf Deep neural nets with a large number of parameters are very powerful machine learning systems 阅读全文

posted @ 2017-11-01 15:25 papering 阅读(718) 评论(0) 推荐(0)

Write Custom Java to Create LZO Files

摘要： https://cwiki.apache.org/confluence/display/Hive/LanguageManual+LZO LanguageManual LZO LanguageManual LZO LanguageManual LZO Skip to end of metadata C 阅读全文

posted @ 2017-11-01 00:03 papering 阅读(490) 评论(0) 推荐(0)

2017年10月31日

Avro schemas are defined with JSON . This facilitates implementation in languages that already have JSON libraries.

摘要： https://avro.apache.org/docs/current/ Introduction Apache Avro™ is a data serialization system. Avro provides: Rich data structures. A compact, fast, 阅读全文

posted @ 2017-10-31 23:45 papering 阅读(248) 评论(0) 推荐(0)

Storage Types and Storage Policies

摘要： https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html Introduction Archival Storage is a solution to decouple gr 阅读全文

posted @ 2017-10-31 23:38 papering 阅读(373) 评论(0) 推荐(0)

splittability A SequenceFile can be split by Hadoop and distributed across map jobs whereas a GZIP file cannot be.

摘要： splittability CompressedStorage CompressedStorage CompressedStorage Skip to end of metadata Created by Confluence Administrator, last modified by Left 阅读全文

posted @ 2017-10-31 23:26 papering 阅读(341) 评论(0) 推荐(0)

This means that only a small number of nodes must be read from disk to retrieve an item.

摘要： http://cis.stvincent.edu/html/tutorials/swd/btree/btree.html Introduction A B-tree is a specialized multiway tree designed especially for use on disk. 阅读全文

posted @ 2017-10-31 21:51 papering 阅读(235) 评论(0) 推荐(0)

Apache Kafka® is a distributed streaming platform. What exactly does that mean?

摘要： https://kafka.apache.org/intro.html 阅读全文

posted @ 2017-10-31 17:03 papering 阅读(192) 评论(0) 推荐(0)

(t,p,o) t:p>=o there cannot be more consumer instances in a consumer group than partitions

摘要： https://kafka.apache.org/intro.html Kafka as a Messaging System How does Kafka's notion of streams compare to a traditional enterprise messaging syste 阅读全文

posted @ 2017-10-31 12:03 papering 阅读(256) 评论(0) 推荐(0)

Goldilocks

摘要： limit 阅读全文

posted @ 2017-10-31 11:22 papering 阅读(162) 评论(0) 推荐(0)

Sqoop

摘要： rmds mapper 阅读全文

posted @ 2017-10-31 11:21 papering 阅读(136) 评论(0) 推荐(0)

Spark SQL includes a cost-based optimizer, columnar storage and code generation to make queries fast.

摘要： https://spark.apache.org/sql/ Performance & Scalability Spark SQL includes a cost-based optimizer, columnar storage and code generation to make querie 阅读全文

posted @ 2017-10-31 00:10 papering 阅读(258) 评论(0) 推荐(0)

2017年10月30日

Sparksql 取代 Hive？

摘要： sparksql hive https://databricks.com/blog/2014/07/01/shark-spark-sql-hive-on-spark-and-the-future-of-sql-on-spark.html https://cwiki.apache.org/conflu 阅读全文

posted @ 2017-10-30 23:43 papering 阅读(720) 评论(0) 推荐(0)

Eliminates these repeated computation in multi aggregations query

摘要： https://github.com/pingcap/tispark/commit/dcca23bfa1aa0c356a4280d82bc8301c0de08318 阅读全文

posted @ 2017-10-30 20:56 papering 阅读(260) 评论(0) 推荐(0)

rule-based optimizer cost-based optimizer

摘要： SQL processing uses the following main components to execute a SQL query: The Parser checks both syntax and semantic analysis. The Optimizer uses cost 阅读全文

posted @ 2017-10-30 20:01 papering 阅读(567) 评论(0) 推荐(0)

Groovy

摘要： https://en.wikipedia.org/wiki/Groovy_(programming_language) Groovy offers simple, consistent syntax for handling lists and maps, reminiscent of Java's 阅读全文

posted @ 2017-10-30 19:54 papering 阅读(230) 评论(0) 推荐(0)

Interface AutoCloseable

摘要： https://docs.oracle.com/javase/tutorial/essential/exceptions/try.html https://docs.oracle.com/javase/8/docs/api/java/lang/AutoCloseable.html An object 阅读全文

posted @ 2017-10-30 17:20 papering 阅读(284) 评论(0) 推荐(0)

throws ? catch checked unchecked

摘要： ThrowableClass Error (unchecked) Exception IOException (checked) RuntimeException (unchecked) 阅读全文

posted @ 2017-10-30 14:58 papering 阅读(215) 评论(0) 推荐(0)

18 ？ 21

摘要： This is for MyHDFSCat = 18+3 阅读全文

posted @ 2017-10-30 10:04 papering 阅读(342) 评论(0) 推荐(0)

2017年10月29日

$wc

摘要： $ wc --helpUsage: wc [OPTION]... [FILE]... or: wc [OPTION]... --files0-from=FPrint newline, word, and byte counts for each FILE, and a total line ifmo 阅读全文

posted @ 2017-10-29 16:43 papering 阅读(280) 评论(0) 推荐(0)

file descriptor 0 1 2 一切皆文件 stdout stderr stdin /dev/null 沉默是金 pipes 禁止输出屏蔽 stdout 和 stderr 输入输出重定向重定向文件描述符

摘要：读取标准输入 #include "thread.h" #include "thread-sync.h" int n, count = 0; mutex_t lk = MUTEX_INIT(); void Tproduce() { while (1) { retry: mutex_lock(&lk); 阅读全文

posted @ 2017-10-29 16:08 papering 阅读(936) 评论(0) 推荐(0)

# $ # root

摘要： # 阅读全文

posted @ 2017-10-29 15:59 papering 阅读(143) 评论(0) 推荐(0)

inode ls -li 显示索引节点

摘要： ls 阅读全文

posted @ 2017-10-29 14:04 papering 阅读(480) 评论(0) 推荐(0)

again shell

摘要： sh UNIX shell Bourne Again Shell 阅读全文

posted @ 2017-10-29 11:27 papering 阅读(207) 评论(0) 推荐(0)

操作系统资源管理 zookeeper yarn 进程管理分布式 yarn诞生背景

摘要： zookeeper 信息保管员 YARN 简介 https://www.ibm.com/developerworks/cn/data/library/bd-yarn-intro/index.html 阅读全文

posted @ 2017-10-29 11:10 papering 阅读(272) 评论(0) 推荐(0)

2017年10月28日

Xen

摘要： https://www.xenproject.org/ The Xen ProjectTM is the leading open source virtualization platform that is powering some of the largest clouds in produc 阅读全文

posted @ 2017-10-28 21:13 papering 阅读(233) 评论(0) 推荐(0)

I/O 复用 multiplexing data race 同步 coroutine 协程

摘要：小结： 1、A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g., read(2)) without blocking. UNIX网络编程卷1：阅读全文

posted @ 2017-10-28 12:58 papering 阅读(372) 评论(0) 推荐(0)

flume topology design . tier num 分层数目

摘要： 32：+：1 x：1 x＜＝8 https://flume.apache.org/FlumeUserGuide.html#flume-topology-design Flume topology design 【拓扑分层原因 0-缓冲1-路由】 The first step in designin 阅读全文

posted @ 2017-10-28 09:26 papering 阅读(266) 评论(0) 推荐(0)

Page (computer memory) Memory segmentation Page table 虚拟地址到物理地址的转换

摘要： A page, memory page, or virtual page is a fixed-length contiguous block of virtual memory, described by a single entry in the page table. It is the sm 阅读全文

posted @ 2017-10-28 02:48 papering 阅读(645) 评论(0) 推荐(0)

2017年10月27日

service oriented architecture 构造分布式计算的应用程序的方法面向服务的架构分解技术

摘要： zh.wikipedia.org/wiki/面向服务的架构【程序功能做为服务】面向服务的体系结构（英语：service-oriented architecture）是构造分布式計算的应用程序的方法。它将应用程序功能作为服务发送给最终用户或者其他服务。它采用开放标准、与软件资源进行交互并采用表示阅读全文

posted @ 2017-10-27 22:31 papering 阅读(369) 评论(0) 推荐(0)

WildFly JBoss 应用程序服务器

摘要： https://en.wikipedia.org/wiki/WildFly 【实现基于面向服务的架构SOA的web应用和服务】 WildFly,[1] formerly known as JBoss AS, or simply JBoss, is an application server auth 阅读全文

posted @ 2017-10-27 22:23 papering 阅读(274) 评论(0) 推荐(0)