02 - ElasticSearch增删改查
作者: elewei
时间: 2017年4月11号
功能: 理解elasticsearch
Elasticsearch使用案例:
- 允许用户在电商系统中搜索产品,使用Elasticsearch保存所有产品目录与信息和提供自动化的推荐功能。
- 想要收集日志或传输信息来进行分析,了解趋势,统计,汇总,使用Logstash来收集,聚合和修剪信息,将信息传入elasticsearch。
- 对用户期望进行告知
- 对大量数据进行调查,分析,可视化,询问最关键的问题,使用kibina建立面板。
1. 运行elasticearch
#启动默认配置信息
cd /opt/elasticsearch-5.3.0/
./bin/elasticsearch
#日志信息
[2017-04-10T15:02:20,012][INFO ][o.e.n.Node ] [] initializing ...
[2017-04-10T15:02:20,222][INFO ][o.e.e.NodeEnvironment ] [WOz1m8N] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [47.9gb], net total_
space [49.9gb], spins? [unknown], types [rootfs]
[2017-04-10T15:02:20,222][INFO ][o.e.e.NodeEnvironment ] [WOz1m8N] heap size [1.9gb], compressed ordinary object pointers [true]
[2017-04-10T15:02:20,223][INFO ][o.e.n.Node ] node name [WOz1m8N] derived from node ID [WOz1m8NxRzGTq1GfROwgqw]; set [node.name] to overri
de
[2017-04-10T15:02:20,223][INFO ][o.e.n.Node ] version[5.3.0], pid[10920], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/3.10.0-514.10.
2.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_121/25.121-b13]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [aggs-matrix-stats]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [ingest-common]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [lang-expression]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [lang-groovy]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [lang-mustache]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [lang-painless]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [percolator]
[2017-04-10T15:02:24,781][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [reindex]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [transport-netty3]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService ] [WOz1m8N] loaded module [transport-netty4]
[2017-04-10T15:02:24,782][INFO ][o.e.p.PluginsService ] [WOz1m8N] no plugins loaded
[2017-04-10T15:02:35,762][INFO ][o.e.n.Node ] initialized
[2017-04-10T15:02:35,797][INFO ][o.e.n.Node ] [WOz1m8N] starting ...
[2017-04-10T15:02:36,486][INFO ][o.e.t.TransportService ] [WOz1m8N] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-04-10T15:02:36,500][WARN ][o.e.b.BootstrapChecks ] [WOz1m8N] max file descriptors [4096] for elasticsearch process is too low, increase to at l
east [65536]
[2017-04-10T15:02:36,500][WARN ][o.e.b.BootstrapChecks ] [WOz1m8N] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least
[262144]
[2017-04-10T15:02:39,616][INFO ][o.e.c.s.ClusterService ] [WOz1m8N] new_master {WOz1m8N}{WOz1m8NxRzGTq1GfROwgqw}{5Ul-LmZaRQWMBri0-gaQxA}{127.0.0.1}{12
7.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-04-10T15:02:39,771][INFO ][o.e.g.GatewayService ] [WOz1m8N] recovered [0] indices into cluster_state
[2017-04-10T15:02:39,784][INFO ][o.e.h.n.Netty4HttpServerTransport] [WOz1m8N] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1
:9200}
[2017-04-10T15:02:39,786][INFO ][o.e.n.Node ] [WOz1m8N] started
#也可以自定cluster和Node名称
./elasticsearch -Ecluster.name=my_cluster_name -Enode.name=my_node_name
2. 与elasticsearch进行交互
- Check your cluster, node, and index health, status, and statistics
- Administer your cluster, node, and index data and metadata
- Perform CRUD (Create, Read, Update, and Delete) and search operations against your indexes
- Execute advanced search operations such as paging, sorting, filtering, scripting, aggregations, and many others
2.1 检查Cluster健康状态
#检查当前状态
curl -XGET 'localhost:9200/_cat/health?v&pretty'
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1491899180 16:26:20 elasticsearch yellow 1 1 16 16 0 0 16 0 - 50.0%
#状态green表示很好
#状态yello表示数据是可以访问的,但是replicas没有分配
#状态red表示一些数据因某些原因不可访问
#查看当前所有节点
curl -XGET 'localhost:9200/_cat/nodes?v&pretty'
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1 5 84 1 0.00 0.04 0.05 mdi * WOz1m8N
2.2 查看所有indices
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana C43KeB4tRneVQ8x2apweNg 1 1 2 0 13.7kb 13.7kb
yellow open logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ 5 1 200 0 530.7kb 530.7kb
yellow open filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A 5 1 700 0 633.5kb 633.5kb
yellow open megacorp OvWH8Pm1QYO40n9d364hkg 5 1 3 0 17kb 17kb
2.3 创建一个Index
#创建一个customer的index
PUT /customer?pretty
#显示
GET /_cat/indices?v
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana C43KeB4tRneVQ8x2apweNg 1 1 2 0 13.7kb 13.7kb
yellow open logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ 5 1 200 0 530.7kb 530.7kb
yellow open filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A 5 1 700 0 633.5kb 633.5kb
yellow open customer UJ38G2sJTpuvP0k7Tnvozw 5 1 0 0 650b 650b
yellow open megacorp OvWH8Pm1QYO40n9d364hkg 5 1 3 0 17kb 17kb
2.4 查询文档
#index一个为external类型的文档,id为1
PUT /customer/external/1?pretty
{
"name" : "John Doe"
}
#回应为
{
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true
}
#查询
GET /customer/external/1?pretty
#回应为
{
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 1,
"found": true,
"_source": {
"name": "John Doe"
}
}
2.5 删除一个index
#删除一个index
DELETE /customer?pretty
#响应为
{
"acknowledged": true
}
#查询
GET /_cat/indices?v
#响应为
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana C43KeB4tRneVQ8x2apweNg 1 1 2 0 13.7kb 13.7kb
yellow open logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ 5 1 200 0 530.7kb 530.7kb
yellow open filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A 5 1 700 0 633.5kb 633.5kb
yellow open megacorp OvWH8Pm1QYO40n9d364hkg 5 1 3 0 17kb 17kb
2. 6 总结
#REST API
PUT /customer
PUT /customer/external/1
{
"name": "John Doe"
}
GET /customer/external/1
DELETE /customer
#curl
curl -XPUT 'localhost:9200/customer?pretty'
curl -XPUT 'localhost:9200/customer/external/1?pretty' -H 'Content-Type: application/json' -d'
{
"name": "John Doe"
}
'
curl -XGET 'localhost:9200/customer/external/1?pretty'
curl -XDELETE 'localhost:9200/customer?pretty'
# 模式
<REST Verb> /<Index>/<Type>/<ID>
3. 修改数据
#index一个表
PUT /customer/external/1?pretty
{
"name" : "John Doe"
}
# 如果执行相同的index操作
# version字段会累计+1
# id字段是一个可选字段,当不写时,elasticsearch会使用一个随机值
{
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 3,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": false
}
#使用POST动作来index一个文档
POST /customer/external?pretty
{
"name": "Jane Doe"
}
#响应为
{
"_index": "customer",
"_type": "external",
"_id": "AVtcRR9y8Lhls8vwECvg", #随机值
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true
}
#注意使用PUT将会出错
No handler found for uri [/customer/external?pretty] and method [PUT]
3.1 更新文档
#更新customer/external/1的名字
POST /customer/external/1/_update?pretty
{
"doc": { "name": "Qi Wei" }
}
#响应为
{
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 7,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
#检查数据
GET /customer/external/1
#增加新的字段
POST /customer/external/1/_update?pretty
{
"doc": { "name": "Jane Doe", "age": 20 }
}
#也可以使用脚本来更新
#ctx._source 表示当前值20
POST /customer/external/1/_update?pretty
{
"script" : "ctx._source.age += 5"
}
#响应结果
{
"_index": "customer",
"_type": "external",
"_id": "1",
"_version": 9,
"found": true,
"_source": {
"name": "Qi Wei",
"age": 25
}
}
3.2 删除文档
#删除一个文档
DELETE /customer/external/2?pretty
#响应为
{
"found": true,
"_index": "customer",
"_type": "external",
"_id": "2",
"_version": 2,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
}
}
3.3 批量执行
# 批量执行index
POST /customer/external/_bulk?pretty
{"index":{"_id":"1"}}
{"name": "John Doe" }
{"index":{"_id":"2"}}
{"name": "Jane Doe" }
#响应为
{
"took": 4,
"errors": false,
"items": [
{
"index": {
"_index": "customer",
"_type": "external",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true,
"status": 201
}
},
{
"index": {
"_index": "customer",
"_type": "external",
"_id": "4",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true,
"status": 201
}
}
]
}
#更新id为1的数据,删除id为2的文档
POST /customer/external/_bulk?pretty
{"update":{"_id":"1"}}
{"doc": { "name": "John Doe becomes Jane Doe" } }
{"delete":{"_id":"2"}}
4. 查询
#插入一个样本数据集accounts.json
{
"account_number": 0,
"balance": 16623,
"firstname": "Bradshaw",
"lastname": "Mckenzie",
"age": 29,
"gender": "F",
"address": "244 Columbus Place",
"employer": "Euron",
"email": "bradshawmckenzie@euron.com",
"city": "Hobucken",
"state": "CO"
}
#保存至当前目录下
curl -H "Content-Type: application/json" -XPOST 'localhost:9200/bank/account/_bulk?pretty&refresh' --data-binary "@accounts.json"
#查看名为bank的index,有1000个文档
curl 'localhost:9200/_cat/indices?v'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open .kibana C43KeB4tRneVQ8x2apweNg 1 1 2 0 13.7kb 13.7kb
yellow open customer 68jOtEGCQ0SDJZ83uI3w0g 5 1 2 0 7.3kb 7.3kb
yellow open logstash-2017.04.10 1r4ZaZplR8aD91DJJeuZvQ 5 1 200 0 530.7kb 530.7kb
yellow open filebeat-2017.04.10 G6rfgE4PSl2JpXAgipuH1A 5 1 700 0 633.5kb 633.5kb
yellow open bank HgUO5xACQHqhWvkoz0qvRQ 5 1 1000 0 640.2kb 640.2kb
yellow open megacorp OvWH8Pm1QYO40n9d364hkg 5 1 3 0 17kb 17kb
4.1 查询
#第一种,使用REST request URL查询
# q=* 告诉elasticsearch匹配所有的index
# sort=account_number:asc 使用account_number字段以升序排列
GET /bank/_search?q=*&sort=account_number:asc&pretty
#响应结果
# took 以微秒为单位
# time_out 表示是否超时
# _shards 搜索了多少分片
# hits.hits 实际的数据(默认前10个)
# hits.sort 排序字段值(没有,默认使用score)
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": null,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "0",
"_score": null,
"_source": {
"account_number": 0,
"balance": 16623,
"firstname": "Bradshaw",
"lastname": "Mckenzie",
"age": 29,
"gender": "F",
"address": "244 Columbus Place",
"employer": "Euron",
"email": "bradshawmckenzie@euron.com",
"city": "Hobucken",
"state": "CO"
},
"sort": [
0
]
},
{
"_index": "bank",
"_type": "account",
"_id": "1",
"_score": null,
"_source": {
"account_number": 1,
"balance": 39225,
"firstname": "Amber",
"lastname": "Duke",
"age": 32,
"gender": "M",
"address": "880 Holmes Lane",
"employer": "Pyrami",
"email": "amberduke@pyrami.com",
"city": "Brogan",
"state": "IL"
},
"sort": [
1
]
},
{
"_index": "bank",
"_type": "account",
"_id": "2",
"_score": null,
"_source": {
"account_number": 2,
"balance": 28838,
"firstname": "Roberta",
"lastname": "Bender",
"age": 22,
"gender": "F",
"address": "560 Kingsway Place",
"employer": "Chillium",
"email": "robertabender@chillium.com",
"city": "Bennett",
"state": "LA"
},
"sort": [
2
]
},
........
{
"_index": "bank",
"_type": "account",
"_id": "9",
"_score": null,
"_source": {
"account_number": 9,
"balance": 24776,
"firstname": "Opal",
"lastname": "Meadows",
"age": 39,
"gender": "M",
"address": "963 Neptune Avenue",
"employer": "Cedward",
"email": "opalmeadows@cedward.com",
"city": "Olney",
"state": "OH"
},
"sort": [
9
]
}
]
}
}
#使用Request Body来进行搜索,结果相同
GET /bank/_search
{
"query": { "match_all": {} },
"sort": [
{ "account_number": "asc" }
]
}
4.2 介绍查询语句
# 执行最简单的查询
GET /bank/_search
{
"query": { "match_all": {} }
}
#响应为
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 1,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "25",
"_score": 1,
"_source": {
"account_number": 25,
"balance": 40540,
"firstname": "Virginia",
"lastname": "Ayala",
"age": 39,
"gender": "F",
"address": "171 Putnam Avenue",
"employer": "Filodyne",
"email": "virginiaayala@filodyne.com",
"city": "Nicholson",
"state": "PA"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "44",
"_score": 1,
"_source": {
"account_number": 44,
"balance": 34487,
"firstname": "Aurelia",
"lastname": "Harding",
"age": 37,
"gender": "M",
"address": "502 Baycliff Terrace",
"employer": "Orbalix",
"email": "aureliaharding@orbalix.com",
"city": "Yardville",
"state": "DE"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "99",
"_score": 1,
"_source": {
"account_number": 99,
"balance": 47159,
"firstname": "Ratliff",
"lastname": "Heath",
"age": 39,
"gender": "F",
"address": "806 Rockwell Place",
"employer": "Zappix",
"email": "ratliffheath@zappix.com",
"city": "Shaft",
"state": "ND"
}
},
..........
{
"_index": "bank",
"_type": "account",
"_id": "208",
"_score": 1,
"_source": {
"account_number": 208,
"balance": 40760,
"firstname": "Garcia",
"lastname": "Hess",
"age": 26,
"gender": "F",
"address": "810 Nostrand Avenue",
"employer": "Quiltigen",
"email": "garciahess@quiltigen.com",
"city": "Brooktrails",
"state": "GA"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "222",
"_score": 1,
"_source": {
"account_number": 222,
"balance": 14764,
"firstname": "Rachelle",
"lastname": "Rice",
"age": 36,
"gender": "M",
"address": "333 Narrows Avenue",
"employer": "Enaut",
"email": "rachellerice@enaut.com",
"city": "Wright",
"state": "AZ"
}
}
]
}
}
# 增加查询条件
GET /bank/_search
{
"query": { "match_all": {} },
"size": 1
}
#结果为
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 1,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "25",
"_score": 1,
"_source": {
"account_number": 25,
"balance": 40540,
"firstname": "Virginia",
"lastname": "Ayala",
"age": 39,
"gender": "F",
"address": "171 Putnam Avenue",
"employer": "Filodyne",
"email": "virginiaayala@filodyne.com",
"city": "Nicholson",
"state": "PA"
}
}
]
}
}
# 增加from条件,如果没设,默认从0开始
GET /bank/_search
{
"query": { "match_all": {} },
"from": 10,
"size": 10
}
# 增加sort,以balance字段降序
GET /bank/_search
{
"query": { "match_all": {} },
"sort": { "balance": { "order": "desc" } }
}
4.3 运行查询
# 只返回两个字段, account_number 和 balance
GET /bank/_search
{
"query": { "match_all": {} },
"_source": ["account_number", "balance"]
}
# 查询结果为
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 1,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "25",
"_score": 1,
"_source": {
"account_number": 25,
"balance": 40540
}
},
.......
{
"_index": "bank",
"_type": "account",
"_id": "222",
"_score": 1,
"_source": {
"account_number": 222,
"balance": 14764
}
}
]
}
}
# 查询只返回account_number为20
GET /bank/_search
{
"query": { "match": { "account_number": 20 } }
}
# 查询结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "20",
"_score": 1,
"_source": {
"account_number": 20,
"balance": 16418,
"firstname": "Elinor",
"lastname": "Ratliff",
"age": 36,
"gender": "M",
"address": "282 Kings Place",
"employer": "Scentric",
"email": "elinorratliff@scentric.com",
"city": "Ribera",
"state": "WA"
}
}
]
}
}
#查询所有在地址中有mail的文档
GET /bank/_search
{
"query": { "match": { "address": "mill" } }
}
# 查询结果
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 4.3100996,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "472",
"_score": 4.3100996,
"_source": {
"account_number": 472,
"balance": 25571,
"firstname": "Lee",
"lastname": "Long",
"age": 32,
"gender": "F",
"address": "288 Mill Street",
"employer": "Comverges",
"email": "leelong@comverges.com",
"city": "Movico",
"state": "MT"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "136",
"_score": 4.2662063,
"_source": {
"account_number": 136,
"balance": 45801,
"firstname": "Winnie",
"lastname": "Holland",
"age": 38,
"gender": "M",
"address": "198 Mill Lane",
"employer": "Neteria",
"email": "winnieholland@neteria.com",
"city": "Urie",
"state": "IL"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "970",
"_score": 3.861861,
"_source": {
"account_number": 970,
"balance": 19648,
"firstname": "Forbes",
"lastname": "Wallace",
"age": 28,
"gender": "M",
"address": "990 Mill Road",
"employer": "Pheast",
"email": "forbeswallace@pheast.com",
"city": "Lopezo",
"state": "AK"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "345",
"_score": 3.861861,
"_source": {
"account_number": 345,
"balance": 9812,
"firstname": "Parker",
"lastname": "Hines",
"age": 38,
"gender": "M",
"address": "715 Mill Avenue",
"employer": "Baluba",
"email": "parkerhines@baluba.com",
"city": "Blackgum",
"state": "KY"
}
}
]
}
}
# 查询所有账户有mill 或 lane
GET /bank/_search
{
"query": { "match": { "address": "mill lane" } }
}
# 查询结果
{
"took": 14,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 19,
"max_score": 7.3900023,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "136",
"_score": 7.3900023,
"_source": {
"account_number": 136,
"balance": 45801,
"firstname": "Winnie",
"lastname": "Holland",
"age": 38,
"gender": "M",
"address": "198 Mill Lane",
"employer": "Neteria",
"email": "winnieholland@neteria.com",
"city": "Urie",
"state": "IL"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "472",
"_score": 4.3100996,
"_source": {
"account_number": 472,
"balance": 25571,
"firstname": "Lee",
"lastname": "Long",
"age": 32,
"gender": "F",
"address": "288 Mill Street",
"employer": "Comverges",
"email": "leelong@comverges.com",
"city": "Movico",
"state": "MT"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "556",
"_score": 3.9074605,
"_source": {
"account_number": 556,
"balance": 36420,
"firstname": "Collier",
"lastname": "Odonnell",
"age": 35,
"gender": "M",
"address": "591 Nolans Lane",
"employer": "Sultraxin",
"email": "collierodonnell@sultraxin.com",
"city": "Fulford",
"state": "MD"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "934",
"_score": 3.9074605,
"_source": {
"account_number": 934,
"balance": 43987,
"firstname": "Freida",
"lastname": "Daniels",
"age": 34,
"gender": "M",
"address": "448 Cove Lane",
"employer": "Vurbo",
"email": "freidadaniels@vurbo.com",
"city": "Snelling",
"state": "NJ"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "970",
"_score": 3.861861,
"_source": {
"account_number": 970,
"balance": 19648,
"firstname": "Forbes",
"lastname": "Wallace",
"age": 28,
"gender": "M",
"address": "990 Mill Road",
"employer": "Pheast",
"email": "forbeswallace@pheast.com",
"city": "Lopezo",
"state": "AK"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "345",
"_score": 3.861861,
"_source": {
"account_number": 345,
"balance": 9812,
"firstname": "Parker",
"lastname": "Hines",
"age": 38,
"gender": "M",
"address": "715 Mill Avenue",
"employer": "Baluba",
"email": "parkerhines@baluba.com",
"city": "Blackgum",
"state": "KY"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "908",
"_score": 3.861861,
"_source": {
"account_number": 908,
"balance": 45975,
"firstname": "Mosley",
"lastname": "Holloway",
"age": 31,
"gender": "M",
"address": "929 Eldert Lane",
"employer": "Anivet",
"email": "mosleyholloway@anivet.com",
"city": "Biehle",
"state": "MS"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "637",
"_score": 3.861861,
"_source": {
"account_number": 637,
"balance": 3169,
"firstname": "Kathy",
"lastname": "Carter",
"age": 27,
"gender": "F",
"address": "410 Jamison Lane",
"employer": "Limage",
"email": "kathycarter@limage.com",
"city": "Ernstville",
"state": "WA"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "449",
"_score": 3.8473392,
"_source": {
"account_number": 449,
"balance": 41950,
"firstname": "Barnett",
"lastname": "Cantrell",
"age": 39,
"gender": "F",
"address": "945 Bedell Lane",
"employer": "Zentility",
"email": "barnettcantrell@zentility.com",
"city": "Swartzville",
"state": "ND"
}
},
{
"_index": "bank",
"_type": "account",
"_id": "742",
"_score": 3.8473392,
"_source": {
"account_number": 742,
"balance": 24765,
"firstname": "Merle",
"lastname": "Wooten",
"age": 26,
"gender": "M",
"address": "317 Pooles Lane",
"employer": "Tropolis",
"email": "merlewooten@tropolis.com",
"city": "Bentley",
"state": "ND"
}
}
]
}
}
# 查询mill lane 字段的账号
GET /bank/_search
{
"query": { "match_phrase": { "address": "mill lane" } }
}
# 查询结果
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 7.3900023,
"hits": [
{
"_index": "bank",
"_type": "account",
"_id": "136",
"_score": 7.3900023,
"_source": {
"account_number": 136,
"balance": 45801,
"firstname": "Winnie",
"lastname": "Holland",
"age": 38,
"gender": "M",
"address": "198 Mill Lane",
"employer": "Neteria",
"email": "winnieholland@neteria.com",
"city": "Urie",
"state": "IL"
}
}
]
}
}
# 使用bool 来 匹配 所有具有 mill 与 lane字段的值
GET /bank/_search
{
"query": {
"bool": {
"must": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
# 查询结果与上一相同
# 使用bool 来 匹配 所有与 mill 或 lane字段的值
GET /bank/_search
{
"query": {
"bool": {
"should": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
# 既不是 mill 也不是 lane
GET /bank/_search
{
"query": {
"bool": {
"must_not": [
{ "match": { "address": "mill" } },
{ "match": { "address": "lane" } }
]
}
}
}
# 返回age 为 40 不生活在id
GET /bank/_search
{
"query": {
"bool": {
"must": [
{ "match": { "age": "40" } }
],
"must_not": [
{ "match": { "state": "ID" } }
]
}
}
}
4.4 执行过滤
# 使用filter过滤出资产在20000 和 30000之间的人
GET /bank/_search
{
"query": {
"bool": {
"must": { "match_all": {} },
"filter": {
"range": {
"balance": {
"gte": 20000,
"lte": 30000
}
}
}
}
}
}
4.5 执行汇聚
# 将所有账号以state来汇聚,返回前10个,降序
GET /bank/_search
{
"size": 0,
"aggs": {
"group_by_state": {
"terms": {
"field": "state.keyword"
}
}
}
}
#返回结果
{
"took": 47,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 0,
"hits": []
},
"aggregations": {
"group_by_state": {
"doc_count_error_upper_bound": 20,
"sum_other_doc_count": 770,
"buckets": [
{
"key": "ID",
"doc_count": 27
},
{
"key": "TX",
"doc_count": 27
},
{
"key": "AL",
"doc_count": 25
},
{
"key": "MD",
"doc_count": 25
},
{
"key": "TN",
"doc_count": 23
},
{
"key": "MA",
"doc_count": 21
},
{
"key": "NC",
"doc_count": 21
},
{
"key": "ND",
"doc_count": 21
},
{
"key": "ME",
"doc_count": 20
},
{
"key": "MO",
"doc_count": 20
}
]
}
}
}
# 计算各地平均资产,以地域来计算,前10
GET /bank/_search
{
"size": 0,
"aggs": {
"group_by_state": {
"terms": {
"field": "state.keyword"
},
"aggs": {
"average_balance": {
"avg": {
"field": "balance"
}
}
}
}
}
}
# 以地域来汇聚,以平均值来降序
GET /bank/_search
{
"size": 0,
"aggs": {
"group_by_state": {
"terms": {
"field": "state.keyword",
"order": {
"average_balance": "desc"
}
},
"aggs": {
"average_balance": {
"avg": {
"field": "balance"
}
}
}
}
}
}
#返回结果
{
"took": 33,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 0,
"hits": []
},
"aggregations": {
"group_by_state": {
"doc_count_error_upper_bound": -1,
"sum_other_doc_count": 918,
"buckets": [
{
"key": "AL",
"doc_count": 6,
"average_balance": {
"value": 41418.166666666664
}
},
{
"key": "SC",
"doc_count": 1,
"average_balance": {
"value": 40019
}
},
{
"key": "AZ",
"doc_count": 10,
"average_balance": {
"value": 36847.4
}
},
{
"key": "VA",
"doc_count": 13,
"average_balance": {
"value": 35418.846153846156
}
},
{
"key": "DE",
"doc_count": 8,
"average_balance": {
"value": 35135.375
}
},
{
"key": "WA",
"doc_count": 7,
"average_balance": {
"value": 34787.142857142855
}
},
{
"key": "ME",
"doc_count": 3,
"average_balance": {
"value": 34539.666666666664
}
},
{
"key": "OK",
"doc_count": 9,
"average_balance": {
"value": 34529.77777777778
}
},
{
"key": "CO",
"doc_count": 13,
"average_balance": {
"value": 33379.769230769234
}
},
{
"key": "MI",
"doc_count": 12,
"average_balance": {
"value": 32905.916666666664
}
}
]
}
}
}
# 以age 20=29, 30-39, 40-49 来汇总
# 以性别,然后得到平均值,不同性别
GET /bank/_search
{
"size": 0,
"aggs": {
"group_by_age": {
"range": {
"field": "age",
"ranges": [
{
"from": 20,
"to": 30
},
{
"from": 30,
"to": 40
},
{
"from": 40,
"to": 50
}
]
},
"aggs": {
"group_by_gender": {
"terms": {
"field": "gender.keyword"
},
"aggs": {
"average_balance": {
"avg": {
"field": "balance"
}
}
}
}
}
}
}
}
#返回结果
{
"took": 42,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1000,
"max_score": 0,
"hits": []
},
"aggregations": {
"group_by_age": {
"buckets": [
{
"key": "20.0-30.0",
"from": 20,
"to": 30,
"doc_count": 451,
"group_by_gender": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "M",
"doc_count": 232,
"average_balance": {
"value": 27374.05172413793
}
},
{
"key": "F",
"doc_count": 219,
"average_balance": {
"value": 25341.260273972603
}
}
]
}
},
{
"key": "30.0-40.0",
"from": 30,
"to": 40,
"doc_count": 504,
"group_by_gender": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "F",
"doc_count": 253,
"average_balance": {
"value": 25670.869565217392
}
},
{
"key": "M",
"doc_count": 251,
"average_balance": {
"value": 24288.239043824702
}
}
]
}
},
{
"key": "40.0-50.0",
"from": 40,
"to": 50,
"doc_count": 45,
"group_by_gender": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "M",
"doc_count": 24,
"average_balance": {
"value": 26474.958333333332
}
},
{
"key": "F",
"doc_count": 21,
"average_balance": {
"value": 27992.571428571428
}
}
]
}
}
]
}
}
}