02 - ElasticSearch增删改查


作者: elewei
时间: 2017年4月11号
功能: 理解elasticsearch


Elasticsearch使用案例:

  • 允许用户在电商系统中搜索产品,使用Elasticsearch保存所有产品目录与信息和提供自动化的推荐功能。
  • 想要收集日志或传输信息来进行分析,了解趋势,统计,汇总,使用Logstash来收集,聚合和修剪信息,将信息传入elasticsearch。
  • 对用户期望进行告知
  • 对大量数据进行调查,分析,可视化,询问最关键的问题,使用kibina建立面板。

1. 运行elasticearch

#启动默认配置信息
cd /opt/elasticsearch-5.3.0/
./bin/elasticsearch

#日志信息
[2017-04-10T15:02:20,012][INFO ][o.e.n.Node               ] [] initializing ...
[2017-04-10T15:02:20,222][INFO ][o.e.e.NodeEnvironment    ] [WOz1m8N] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [47.9gb], net total_
space [49.9gb], spins? [unknown], types [rootfs]
[2017-04-10T15:02:20,222][INFO ][o.e.e.NodeEnvironment    ] [WOz1m8N] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-10T15:02:20,223][INFO ][o.e.n.Node               ] node name [WOz1m8N] derived from node ID [WOz1m8NxRzGTq1GfROwgqw]; set [node.name] to overri
de
[2017-04-10T15:02:20,223][INFO ][o.e.n.Node               ] version[5.3.0], pid[10920], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/3.10.0-514.10.
2.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [aggs-matrix-stats]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [ingest-common]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [lang-expression]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [lang-groovy]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [lang-mustache]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [lang-painless]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [percolator]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [reindex]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [transport-netty3]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService     ] [WOz1m8N] loaded module [transport-netty4]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService     ] [WOz1m8N] no plugins loaded
[2017-04-10T15:02:35,762][INFO ][o.e.n.Node               ] initialized
[2017-04-10T15:02:35,797][INFO ][o.e.n.Node               ] [WOz1m8N] starting ...
[2017-04-10T15:02:36,486][INFO ][o.e.t.TransportService   ] [WOz1m8N] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-04-10T15:02:36,500][WARN ][o.e.b.BootstrapChecks    ] [WOz1m8N] max file descriptors [4096] for elasticsearch process is too low, increase to at l
east [65536]
[2017-04-10T15:02:36,500][WARN ][o.e.b.BootstrapChecks    ] [WOz1m8N] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least
 [262144]
[2017-04-10T15:02:39,616][INFO ][o.e.c.s.ClusterService   ] [WOz1m8N] new_master {WOz1m8N}{WOz1m8NxRzGTq1GfROwgqw}{5Ul-LmZaRQWMBri0-gaQxA}{127.0.0.1}{12
7.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-10T15:02:39,771][INFO ][o.e.g.GatewayService     ] [WOz1m8N] recovered [0] indices into cluster_state
[2017-04-10T15:02:39,784][INFO ][o.e.h.n.Netty4HttpServerTransport] [WOz1m8N] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1
:9200}
[2017-04-10T15:02:39,786][INFO ][o.e.n.Node               ] [WOz1m8N] started

#也可以自定cluster和Node名称
./elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name

2. 与elasticsearch进行交互

  • Check your cluster, node, and index health, status, and statistics
  • Administer your cluster, node, and index data and metadata
  • Perform CRUD (Create, Read, Update, and Delete) and search operations against your indexes
  • Execute advanced search operations such as paging, sorting, filtering, scripting, aggregations, and many others

2.1 检查Cluster健康状态

#检查当前状态
curl -XGET 'localhost:9200/_cat/health?v&pretty'
epoch      timestamp cluster       status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1491899180 16:26:20  elasticsearch yellow          1         1     16  16    0    0       16             0                  -                 50.0%

#状态green表示很好
#状态yello表示数据是可以访问的,但是replicas没有分配
#状态red表示一些数据因某些原因不可访问

#查看当前所有节点
curl -XGET 'localhost:9200/_cat/nodes?v&pretty'
ip        heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1            5          84   1    0.00    0.04     0.05 mdi       *      WOz1m8N

2.2 查看所有indices

curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index               uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana             C43KeB4tRneVQ8x2apweNg   1   1          2            0     13.7kb         13.7kb
yellow open   logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ   5   1        200            0    530.7kb        530.7kb
yellow open   filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A   5   1        700            0    633.5kb        633.5kb
yellow open   megacorp            OvWH8Pm1QYO40n9d364hkg   5   1          3            0       17kb           17kb

2.3 创建一个Index

#创建一个customer的index
PUT /customer?pretty

#显示
GET /_cat/indices?v

health status index               uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana             C43KeB4tRneVQ8x2apweNg   1   1          2            0     13.7kb         13.7kb
yellow open   logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ   5   1        200            0    530.7kb        530.7kb
yellow open   filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A   5   1        700            0    633.5kb        633.5kb
yellow open   customer            UJ38G2sJTpuvP0k7Tnvozw   5   1          0            0       650b           650b
yellow open   megacorp            OvWH8Pm1QYO40n9d364hkg   5   1          3            0       17kb           17kb

2.4 查询文档

#index一个为external类型的文档,id为1
PUT /customer/external/1?pretty
{
	"name" : "John Doe"
}

#回应为
{
	"_index": "customer",
	"_type": "external",
	"_id": "1",
	"_version": 1,
	"result": "created",
	"_shards": {
		"total": 2,
		"successful": 1,
		"failed": 0
	},
	"created": true
}

#查询
GET /customer/external/1?pretty

#回应为
{
  "_index": "customer",
  "_type": "external",
  "_id": "1",
  "_version": 1,
  "found": true,
  "_source": {
    "name": "John Doe"
  }
}

2.5 删除一个index

#删除一个index
DELETE /customer?pretty

#响应为
{
  "acknowledged": true
}

#查询
GET /_cat/indices?v

#响应为
health status index               uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana             C43KeB4tRneVQ8x2apweNg   1   1          2            0     13.7kb         13.7kb
yellow open   logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ   5   1        200            0    530.7kb        530.7kb
yellow open   filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A   5   1        700            0    633.5kb        633.5kb
yellow open   megacorp            OvWH8Pm1QYO40n9d364hkg   5   1          3            0       17kb           17kb

2. 6 总结

#REST API
PUT /customer
PUT /customer/external/1
{
  "name": "John Doe"
}
GET /customer/external/1
DELETE /customer

#curl 
curl -XPUT 'localhost:9200/customer?pretty'
curl -XPUT 'localhost:9200/customer/external/1?pretty' -H 'Content-Type: application/json' -d'
{
  "name": "John Doe"
}
'
curl -XGET 'localhost:9200/customer/external/1?pretty'
curl -XDELETE 'localhost:9200/customer?pretty'

# 模式
<REST Verb> /<Index>/<Type>/<ID>

3. 修改数据

#index一个表
PUT /customer/external/1?pretty
{
  "name" : "John Doe"
}

# 如果执行相同的index操作
# version字段会累计+1
# id字段是一个可选字段,当不写时,elasticsearch会使用一个随机值

{
  "_index": "customer",
  "_type": "external",
  "_id": "1",
  "_version": 3,
  "result": "updated",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "created": false
}

#使用POST动作来index一个文档
POST /customer/external?pretty
{
  "name": "Jane Doe"
}

#响应为
{
  "_index": "customer",
  "_type": "external",
  "_id": "AVtcRR9y8Lhls8vwECvg",   #随机值
  "_version": 1,
  "result": "created",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  },
  "created": true
}

#注意使用PUT将会出错
No handler found for uri [/customer/external?pretty] and method [PUT]

3.1 更新文档

#更新customer/external/1的名字
POST /customer/external/1/_update?pretty
{
  "doc": { "name": "Qi Wei" }
}

#响应为
{
  "_index": "customer",
  "_type": "external",
  "_id": "1",
  "_version": 7,
  "result": "updated",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  }
}

#检查数据
GET /customer/external/1

#增加新的字段
POST /customer/external/1/_update?pretty
{
  "doc": { "name": "Jane Doe", "age": 20 }
}

#也可以使用脚本来更新
#ctx._source 表示当前值20

POST /customer/external/1/_update?pretty
{
  "script" : "ctx._source.age += 5"
}

#响应结果
{
  "_index": "customer",
  "_type": "external",
  "_id": "1",
  "_version": 9,
  "found": true,
  "_source": {
    "name": "Qi Wei",
    "age": 25
  }
}

3.2 删除文档

#删除一个文档
DELETE /customer/external/2?pretty

#响应为
{
  "found": true,
  "_index": "customer",
  "_type": "external",
  "_id": "2",
  "_version": 2,
  "result": "deleted",
  "_shards": {
    "total": 2,
    "successful": 1,
    "failed": 0
  }
}

3.3 批量执行

# 批量执行index
POST /customer/external/_bulk?pretty
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }

#响应为
{
  "took": 4,
  "errors": false,
  "items": [
    {
      "index": {
        "_index": "customer",
        "_type": "external",
        "_id": "3",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "created": true,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "customer",
        "_type": "external",
        "_id": "4",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "created": true,
        "status": 201
      }
    }
  ]
}


#更新id为1的数据,删除id为2的文档
POST /customer/external/_bulk?pretty
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}

4. 查询

#插入一个样本数据集accounts.json

{
    "account_number": 0,
    "balance": 16623,
    "firstname": "Bradshaw",
    "lastname": "Mckenzie",
    "age": 29,
    "gender": "F",
    "address": "244 Columbus Place",
    "employer": "Euron",
    "email": "bradshawmckenzie@euron.com",
    "city": "Hobucken",
    "state": "CO"
}

#保存至当前目录下
curl -H "Content-Type: application/json" -XPOST 'localhost:9200/bank/account/_bulk?pretty&refresh' --data-binary "@accounts.json"

#查看名为bank的index,有1000个文档
curl 'localhost:9200/_cat/indices?v'  
health status index               uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .kibana             C43KeB4tRneVQ8x2apweNg   1   1          2            0     13.7kb         13.7kb
yellow open   customer            68jOtEGCQ0SDJZ83uI3w0g   5   1          2            0      7.3kb          7.3kb
yellow open   logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ   5   1        200            0    530.7kb        530.7kb
yellow open   filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A   5   1        700            0    633.5kb        633.5kb
yellow open   bank                HgUO5xACQHqhWvkoz0qvRQ   5   1       1000            0    640.2kb        640.2kb
yellow open   megacorp            OvWH8Pm1QYO40n9d364hkg   5   1          3            0       17kb           17kb

4.1 查询

#第一种,使用REST request URL查询
# q=* 告诉elasticsearch匹配所有的index
# sort=account_number:asc 使用account_number字段以升序排列

GET /bank/_search?q=*&sort=account_number:asc&pretty


#响应结果
# took 以微秒为单位
# time_out 表示是否超时
# _shards 搜索了多少分片
# hits.hits 实际的数据(默认前10个)
# hits.sort 排序字段值(没有,默认使用score)
{
  "took": 7,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": null,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "0",
        "_score": null,
        "_source": {
          "account_number": 0,
          "balance": 16623,
          "firstname": "Bradshaw",
          "lastname": "Mckenzie",
          "age": 29,
          "gender": "F",
          "address": "244 Columbus Place",
          "employer": "Euron",
          "email": "bradshawmckenzie@euron.com",
          "city": "Hobucken",
          "state": "CO"
        },
        "sort": [
          0
        ]
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "1",
        "_score": null,
        "_source": {
          "account_number": 1,
          "balance": 39225,
          "firstname": "Amber",
          "lastname": "Duke",
          "age": 32,
          "gender": "M",
          "address": "880 Holmes Lane",
          "employer": "Pyrami",
          "email": "amberduke@pyrami.com",
          "city": "Brogan",
          "state": "IL"
        },
        "sort": [
          1
        ]
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "2",
        "_score": null,
        "_source": {
          "account_number": 2,
          "balance": 28838,
          "firstname": "Roberta",
          "lastname": "Bender",
          "age": 22,
          "gender": "F",
          "address": "560 Kingsway Place",
          "employer": "Chillium",
          "email": "robertabender@chillium.com",
          "city": "Bennett",
          "state": "LA"
        },
        "sort": [
          2
        ]
      },

	  ........

      {
        "_index": "bank",
        "_type": "account",
        "_id": "9",
        "_score": null,
        "_source": {
          "account_number": 9,
          "balance": 24776,
          "firstname": "Opal",
          "lastname": "Meadows",
          "age": 39,
          "gender": "M",
          "address": "963 Neptune Avenue",
          "employer": "Cedward",
          "email": "opalmeadows@cedward.com",
          "city": "Olney",
          "state": "OH"
        },
        "sort": [
          9
        ]
      }
    ]
  }
}

#使用Request Body来进行搜索,结果相同
GET /bank/_search
{
  "query": { "match_all": {} },
  "sort": [
    { "account_number": "asc" }
  ]
}

4.2 介绍查询语句

# 执行最简单的查询
GET /bank/_search
{
  "query": { "match_all": {} }
}

#响应为
{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 1,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "25",
        "_score": 1,
        "_source": {
          "account_number": 25,
          "balance": 40540,
          "firstname": "Virginia",
          "lastname": "Ayala",
          "age": 39,
          "gender": "F",
          "address": "171 Putnam Avenue",
          "employer": "Filodyne",
          "email": "virginiaayala@filodyne.com",
          "city": "Nicholson",
          "state": "PA"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "44",
        "_score": 1,
        "_source": {
          "account_number": 44,
          "balance": 34487,
          "firstname": "Aurelia",
          "lastname": "Harding",
          "age": 37,
          "gender": "M",
          "address": "502 Baycliff Terrace",
          "employer": "Orbalix",
          "email": "aureliaharding@orbalix.com",
          "city": "Yardville",
          "state": "DE"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "99",
        "_score": 1,
        "_source": {
          "account_number": 99,
          "balance": 47159,
          "firstname": "Ratliff",
          "lastname": "Heath",
          "age": 39,
          "gender": "F",
          "address": "806 Rockwell Place",
          "employer": "Zappix",
          "email": "ratliffheath@zappix.com",
          "city": "Shaft",
          "state": "ND"
        }
      },
	..........
      {
        "_index": "bank",
        "_type": "account",
        "_id": "208",
        "_score": 1,
        "_source": {
          "account_number": 208,
          "balance": 40760,
          "firstname": "Garcia",
          "lastname": "Hess",
          "age": 26,
          "gender": "F",
          "address": "810 Nostrand Avenue",
          "employer": "Quiltigen",
          "email": "garciahess@quiltigen.com",
          "city": "Brooktrails",
          "state": "GA"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "222",
        "_score": 1,
        "_source": {
          "account_number": 222,
          "balance": 14764,
          "firstname": "Rachelle",
          "lastname": "Rice",
          "age": 36,
          "gender": "M",
          "address": "333 Narrows Avenue",
          "employer": "Enaut",
          "email": "rachellerice@enaut.com",
          "city": "Wright",
          "state": "AZ"
        }
      }
    ]
  }
}


# 增加查询条件
GET /bank/_search
{
  "query": { "match_all": {} },
  "size": 1
}

#结果为
{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 1,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "25",
        "_score": 1,
        "_source": {
          "account_number": 25,
          "balance": 40540,
          "firstname": "Virginia",
          "lastname": "Ayala",
          "age": 39,
          "gender": "F",
          "address": "171 Putnam Avenue",
          "employer": "Filodyne",
          "email": "virginiaayala@filodyne.com",
          "city": "Nicholson",
          "state": "PA"
        }
      }
    ]
  }
}

# 增加from条件,如果没设,默认从0开始
GET /bank/_search
{
  "query": { "match_all": {} },
  "from": 10,
  "size": 10
}

# 增加sort,以balance字段降序
GET /bank/_search
{
  "query": { "match_all": {} },
  "sort": { "balance": { "order": "desc" } }
}

4.3 运行查询

# 只返回两个字段, account_number 和 balance
GET /bank/_search
{
  "query": { "match_all": {} },
  "_source": ["account_number", "balance"]
}

# 查询结果为
{
  "took": 11,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 1,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "25",
        "_score": 1,
        "_source": {
          "account_number": 25,
          "balance": 40540
        }
      },
	.......
      {
        "_index": "bank",
        "_type": "account",
        "_id": "222",
        "_score": 1,
        "_source": {
          "account_number": 222,
          "balance": 14764
        }
      }
    ]
  }
}

# 查询只返回account_number为20
GET /bank/_search
{
  "query": { "match": { "account_number": 20 } }
}

# 查询结果
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 1,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "20",
        "_score": 1,
        "_source": {
          "account_number": 20,
          "balance": 16418,
          "firstname": "Elinor",
          "lastname": "Ratliff",
          "age": 36,
          "gender": "M",
          "address": "282 Kings Place",
          "employer": "Scentric",
          "email": "elinorratliff@scentric.com",
          "city": "Ribera",
          "state": "WA"
        }
      }
    ]
  }
}

#查询所有在地址中有mail的文档
GET /bank/_search
{
  "query": { "match": { "address": "mill" } }
}

# 查询结果
{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 4,
    "max_score": 4.3100996,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "472",
        "_score": 4.3100996,
        "_source": {
          "account_number": 472,
          "balance": 25571,
          "firstname": "Lee",
          "lastname": "Long",
          "age": 32,
          "gender": "F",
          "address": "288 Mill Street",
          "employer": "Comverges",
          "email": "leelong@comverges.com",
          "city": "Movico",
          "state": "MT"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "136",
        "_score": 4.2662063,
        "_source": {
          "account_number": 136,
          "balance": 45801,
          "firstname": "Winnie",
          "lastname": "Holland",
          "age": 38,
          "gender": "M",
          "address": "198 Mill Lane",
          "employer": "Neteria",
          "email": "winnieholland@neteria.com",
          "city": "Urie",
          "state": "IL"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "970",
        "_score": 3.861861,
        "_source": {
          "account_number": 970,
          "balance": 19648,
          "firstname": "Forbes",
          "lastname": "Wallace",
          "age": 28,
          "gender": "M",
          "address": "990 Mill Road",
          "employer": "Pheast",
          "email": "forbeswallace@pheast.com",
          "city": "Lopezo",
          "state": "AK"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "345",
        "_score": 3.861861,
        "_source": {
          "account_number": 345,
          "balance": 9812,
          "firstname": "Parker",
          "lastname": "Hines",
          "age": 38,
          "gender": "M",
          "address": "715 Mill Avenue",
          "employer": "Baluba",
          "email": "parkerhines@baluba.com",
          "city": "Blackgum",
          "state": "KY"
        }
      }
    ]
  }
}


# 查询所有账户有mill 或 lane
GET /bank/_search
{
  "query": { "match": { "address": "mill lane" } }
}

# 查询结果
{
  "took": 14,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 19,
    "max_score": 7.3900023,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "136",
        "_score": 7.3900023,
        "_source": {
          "account_number": 136,
          "balance": 45801,
          "firstname": "Winnie",
          "lastname": "Holland",
          "age": 38,
          "gender": "M",
          "address": "198 Mill Lane",
          "employer": "Neteria",
          "email": "winnieholland@neteria.com",
          "city": "Urie",
          "state": "IL"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "472",
        "_score": 4.3100996,
        "_source": {
          "account_number": 472,
          "balance": 25571,
          "firstname": "Lee",
          "lastname": "Long",
          "age": 32,
          "gender": "F",
          "address": "288 Mill Street",
          "employer": "Comverges",
          "email": "leelong@comverges.com",
          "city": "Movico",
          "state": "MT"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "556",
        "_score": 3.9074605,
        "_source": {
          "account_number": 556,
          "balance": 36420,
          "firstname": "Collier",
          "lastname": "Odonnell",
          "age": 35,
          "gender": "M",
          "address": "591 Nolans Lane",
          "employer": "Sultraxin",
          "email": "collierodonnell@sultraxin.com",
          "city": "Fulford",
          "state": "MD"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "934",
        "_score": 3.9074605,
        "_source": {
          "account_number": 934,
          "balance": 43987,
          "firstname": "Freida",
          "lastname": "Daniels",
          "age": 34,
          "gender": "M",
          "address": "448 Cove Lane",
          "employer": "Vurbo",
          "email": "freidadaniels@vurbo.com",
          "city": "Snelling",
          "state": "NJ"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "970",
        "_score": 3.861861,
        "_source": {
          "account_number": 970,
          "balance": 19648,
          "firstname": "Forbes",
          "lastname": "Wallace",
          "age": 28,
          "gender": "M",
          "address": "990 Mill Road",
          "employer": "Pheast",
          "email": "forbeswallace@pheast.com",
          "city": "Lopezo",
          "state": "AK"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "345",
        "_score": 3.861861,
        "_source": {
          "account_number": 345,
          "balance": 9812,
          "firstname": "Parker",
          "lastname": "Hines",
          "age": 38,
          "gender": "M",
          "address": "715 Mill Avenue",
          "employer": "Baluba",
          "email": "parkerhines@baluba.com",
          "city": "Blackgum",
          "state": "KY"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "908",
        "_score": 3.861861,
        "_source": {
          "account_number": 908,
          "balance": 45975,
          "firstname": "Mosley",
          "lastname": "Holloway",
          "age": 31,
          "gender": "M",
          "address": "929 Eldert Lane",
          "employer": "Anivet",
          "email": "mosleyholloway@anivet.com",
          "city": "Biehle",
          "state": "MS"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "637",
        "_score": 3.861861,
        "_source": {
          "account_number": 637,
          "balance": 3169,
          "firstname": "Kathy",
          "lastname": "Carter",
          "age": 27,
          "gender": "F",
          "address": "410 Jamison Lane",
          "employer": "Limage",
          "email": "kathycarter@limage.com",
          "city": "Ernstville",
          "state": "WA"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "449",
        "_score": 3.8473392,
        "_source": {
          "account_number": 449,
          "balance": 41950,
          "firstname": "Barnett",
          "lastname": "Cantrell",
          "age": 39,
          "gender": "F",
          "address": "945 Bedell Lane",
          "employer": "Zentility",
          "email": "barnettcantrell@zentility.com",
          "city": "Swartzville",
          "state": "ND"
        }
      },
      {
        "_index": "bank",
        "_type": "account",
        "_id": "742",
        "_score": 3.8473392,
        "_source": {
          "account_number": 742,
          "balance": 24765,
          "firstname": "Merle",
          "lastname": "Wooten",
          "age": 26,
          "gender": "M",
          "address": "317 Pooles Lane",
          "employer": "Tropolis",
          "email": "merlewooten@tropolis.com",
          "city": "Bentley",
          "state": "ND"
        }
      }
    ]
  }
}


# 查询mill lane 字段的账号
GET /bank/_search
{
  "query": { "match_phrase": { "address": "mill lane" } }
}

# 查询结果
{
  "took": 6,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1,
    "max_score": 7.3900023,
    "hits": [
      {
        "_index": "bank",
        "_type": "account",
        "_id": "136",
        "_score": 7.3900023,
        "_source": {
          "account_number": 136,
          "balance": 45801,
          "firstname": "Winnie",
          "lastname": "Holland",
          "age": 38,
          "gender": "M",
          "address": "198 Mill Lane",
          "employer": "Neteria",
          "email": "winnieholland@neteria.com",
          "city": "Urie",
          "state": "IL"
        }
      }
    ]
  }
}

# 使用bool 来 匹配 所有具有 mill 与 lane字段的值
GET /bank/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}

# 查询结果与上一相同

# 使用bool 来 匹配 所有与 mill 或 lane字段的值
GET /bank/_search
{
  "query": {
    "bool": {
      "should": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}

# 既不是 mill 也不是 lane
GET /bank/_search
{
  "query": {
    "bool": {
      "must_not": [
        { "match": { "address": "mill" } },
        { "match": { "address": "lane" } }
      ]
    }
  }
}

# 返回age 为 40 不生活在id
GET /bank/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "age": "40" } }
      ],
      "must_not": [
        { "match": { "state": "ID" } }
      ]
    }
  }
}

4.4 执行过滤

# 使用filter过滤出资产在20000 和 30000之间的人
GET /bank/_search
{
  "query": {
    "bool": {
      "must": { "match_all": {} },
      "filter": {
        "range": {
          "balance": {
            "gte": 20000,
            "lte": 30000
          }
        }
      }
    }
  }
}

4.5 执行汇聚

# 将所有账号以state来汇聚,返回前10个,降序
GET /bank/_search
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state.keyword"
      }
    }
  }
}	


#返回结果
{
  "took": 47,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_by_state": {
      "doc_count_error_upper_bound": 20,
      "sum_other_doc_count": 770,
      "buckets": [
        {
          "key": "ID",
          "doc_count": 27
        },
        {
          "key": "TX",
          "doc_count": 27
        },
        {
          "key": "AL",
          "doc_count": 25
        },
        {
          "key": "MD",
          "doc_count": 25
        },
        {
          "key": "TN",
          "doc_count": 23
        },
        {
          "key": "MA",
          "doc_count": 21
        },
        {
          "key": "NC",
          "doc_count": 21
        },
        {
          "key": "ND",
          "doc_count": 21
        },
        {
          "key": "ME",
          "doc_count": 20
        },
        {
          "key": "MO",
          "doc_count": 20
        }
      ]
    }
  }
}

# 计算各地平均资产,以地域来计算,前10
GET /bank/_search
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state.keyword"
      },
      "aggs": {
        "average_balance": {
          "avg": {
            "field": "balance"
          }
        }
      }
    }
  }
}

# 以地域来汇聚,以平均值来降序
GET /bank/_search
{
  "size": 0,
  "aggs": {
    "group_by_state": {
      "terms": {
        "field": "state.keyword",
        "order": {
          "average_balance": "desc"
        }
      },
      "aggs": {
        "average_balance": {
          "avg": {
            "field": "balance"
          }
        }
      }
    }
  }
}

#返回结果
{
  "took": 33,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_by_state": {
      "doc_count_error_upper_bound": -1,
      "sum_other_doc_count": 918,
      "buckets": [
        {
          "key": "AL",
          "doc_count": 6,
          "average_balance": {
            "value": 41418.166666666664
          }
        },
        {
          "key": "SC",
          "doc_count": 1,
          "average_balance": {
            "value": 40019
          }
        },
        {
          "key": "AZ",
          "doc_count": 10,
          "average_balance": {
            "value": 36847.4
          }
        },
        {
          "key": "VA",
          "doc_count": 13,
          "average_balance": {
            "value": 35418.846153846156
          }
        },
        {
          "key": "DE",
          "doc_count": 8,
          "average_balance": {
            "value": 35135.375
          }
        },
        {
          "key": "WA",
          "doc_count": 7,
          "average_balance": {
            "value": 34787.142857142855
          }
        },
        {
          "key": "ME",
          "doc_count": 3,
          "average_balance": {
            "value": 34539.666666666664
          }
        },
        {
          "key": "OK",
          "doc_count": 9,
          "average_balance": {
            "value": 34529.77777777778
          }
        },
        {
          "key": "CO",
          "doc_count": 13,
          "average_balance": {
            "value": 33379.769230769234
          }
        },
        {
          "key": "MI",
          "doc_count": 12,
          "average_balance": {
            "value": 32905.916666666664
          }
        }
      ]
    }
  }
}


# 以age 20=29, 30-39, 40-49 来汇总
# 以性别,然后得到平均值,不同性别
GET /bank/_search
{
  "size": 0,
  "aggs": {
    "group_by_age": {
      "range": {
        "field": "age",
        "ranges": [
          {
            "from": 20,
            "to": 30
          },
          {
            "from": 30,
            "to": 40
          },
          {
            "from": 40,
            "to": 50
          }
        ]
      },
      "aggs": {
        "group_by_gender": {
          "terms": {
            "field": "gender.keyword"
          },
          "aggs": {
            "average_balance": {
              "avg": {
                "field": "balance"
              }
            }
          }
        }
      }
    }
  }
}

#返回结果
{
  "took": 42,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 1000,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "group_by_age": {
      "buckets": [
        {
          "key": "20.0-30.0",
          "from": 20,
          "to": 30,
          "doc_count": 451,
          "group_by_gender": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "M",
                "doc_count": 232,
                "average_balance": {
                  "value": 27374.05172413793
                }
              },
              {
                "key": "F",
                "doc_count": 219,
                "average_balance": {
                  "value": 25341.260273972603
                }
              }
            ]
          }
        },
        {
          "key": "30.0-40.0",
          "from": 30,
          "to": 40,
          "doc_count": 504,
          "group_by_gender": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "F",
                "doc_count": 253,
                "average_balance": {
                  "value": 25670.869565217392
                }
              },
              {
                "key": "M",
                "doc_count": 251,
                "average_balance": {
                  "value": 24288.239043824702
                }
              }
            ]
          }
        },
        {
          "key": "40.0-50.0",
          "from": 40,
          "to": 50,
          "doc_count": 45,
          "group_by_gender": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
              {
                "key": "M",
                "doc_count": 24,
                "average_balance": {
                  "value": 26474.958333333332
                }
              },
              {
                "key": "F",
                "doc_count": 21,
                "average_balance": {
                  "value": 27992.571428571428
                }
              }
            ]
          }
        }
      ]
    }
  }
}
posted @ 2017-04-11 20:08  elewei  阅读(655)  评论(0编辑  收藏  举报