微服务框架

【SpringCloud+RabbitMQ+Docker+Redis+搜索+分布式，系统详解springcloud微服务技术栈课程|黑马程序员Java微服务】

SpringCloud微服务架构

文章目录

• • • 微服务框架
    • SpringCloud微服务架构
    • 23 搜索结果处理
    • • 23.2 分页
      • • 23.2.1 分页
        • 23.2.2 深度分页问题
        • 23.2.3 深度分页解决方案
        • 23.2.4 总结

23 搜索结果处理

23.2 分页

23.2.1 分页

elasticsearch 默认情况下只返回top10的数据。而如果要查询更多数据就需要修改分页参数了。

elasticsearch中通过修改from、size参数来控制要返回的分页结果：

之前默认查询所有文档，默认显示10条，就是因为ES 底层有一个默认的分页参数

【语法】

GET /hotel/_search
{"query": {"match_all": {}},"from": 990, // 分页开始的位置，默认为0"size": 10, // 期望获取的文档总数"sort": [{"price": "asc"}]
}

试试

# 分页查询
GET /hotel/_search
{"query": {"match_all": {}},"sort": [{"price": "asc"}],"from": 0,"size": 3
}

查询第一页，每页三条

运行结果

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 201,"relation" : "eq"},"max_score" : null,"hits" : [{"_index" : "hotel","_type" : "_doc","_id" : "197837109","_score" : null,"_source" : {"address" : "布吉镇深惠路龙珠商城","brand" : "如家","business" : "布吉/深圳东站","city" : "深圳","id" : 197837109,"location" : "22.602482, 114.123284","name" : "如家酒店·neo(深圳龙岗大道布吉地铁站店)","pic" : "https://m.tuniucdn.com/fb2/t1/G6/M00/25/58/Cii-TF3PFZOIA7jwAAKInGFN4xgAAEVbAGeP4AAAoi0485_w200_h200_c1_t0.jpg","price" : 127,"score" : 43,"starName" : "二钻"},"sort" : [127]},{"_index" : "hotel","_type" : "_doc","_id" : "2316304","_score" : null,"_source" : {"address" : "龙岗街道龙岗墟社区龙平东路62号","brand" : "如家","business" : "龙岗中心区/大运新城","city" : "深圳","id" : 2316304,"location" : "22.730828, 114.278337","name" : "如家酒店(深圳双龙地铁站店)","pic" : "https://m.tuniucdn.com/fb3/s1/2n9c/4AzEoQ44awd1D2g95a6XDtJf3dkw_w200_h200_c1_t0.jpg","price" : 135,"score" : 45,"starName" : "二钻"},"sort" : [135]},{"_index" : "hotel","_type" : "_doc","_id" : "1630005459","_score" : null,"_source" : {"address" : "罗湖区宝安南路2078号深港豪苑（与红桂路交汇处）","brand" : "7天酒店","business" : "","city" : "深圳","id" : 1630005459,"location" : "22.550341, 114.10965","name" : "7天连锁酒店（深圳地王大厦红桂路店）（原红桂路店）","pic" : "https://m.tuniucdn.com/fb2/t1/G2/M00/EA/18/Cii-T1k1KaGIIkQVAAD4fD_T3FcAALTtABiCJ8AAPiU164_w200_h200_c1_t0.jpg","price" : 143,"score" : 39,"starName" : "二钻"},"sort" : [143]}]}
}

看第二页

# 分页查询
GET /hotel/_search
{"query": {"match_all": {}},"sort": [{"price": "asc"}],"from": 3,"size": 3
}

运行结果

{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 201,"relation" : "eq"},"max_score" : null,"hits" : [{"_index" : "hotel","_type" : "_doc","_id" : "541619","_score" : null,"_source" : {"address" : "莘庄镇莘浜路172号","brand" : "如家","business" : "莘庄工业区","city" : "上海","id" : 541619,"location" : "31.105797, 121.37755","name" : "如家酒店(上海莘庄地铁站龙之梦商业广场店)","pic" : "https://m.tuniucdn.com/fb3/s1/2n9c/3mKs3jETvJDj3dDdkRB9UyLLvPna_w200_h200_c1_t0.jpg","price" : 149,"score" : 44,"starName" : "二钻"},"sort" : [149]},{"_index" : "hotel","_type" : "_doc","_id" : "1400304687","_score" : null,"_source" : {"address" : "龙岗大道横岗段4004号","brand" : "如家","business" : "龙岗中心区/大运新城","city" : "深圳","id" : 1400304687,"location" : "22.642629, 114.202799","name" : "如家酒店(深圳横岗地铁站新马商贸城店)","pic" : "https://m.tuniucdn.com/fb2/t1/G6/M00/25/5A/Cii-TF3PFkiIb27dAAEqdDcKl3YAAEViQGVWY0AASqM960_w200_h200_c1_t0.jpg","price" : 149,"score" : 43,"starName" : "二钻"},"sort" : [149]},{"_index" : "hotel","_type" : "_doc","_id" : "728415","_score" : null,"_source" : {"address" : "晒布路67号","brand" : "如家","business" : "东门商业区","city" : "深圳","id" : 728415,"location" : "22.550183, 114.120771","name" : "如家酒店·neo(深圳东门步行街晒布地铁站店)","pic" : "https://m.tuniucdn.com/fb2/t1/G6/M00/25/57/Cii-U13PFNWISSnQAAEpTtoilsQAAEVWgEvur8AASlm647_w200_h200_c1_t0.jpg","price" : 152,"score" : 46,"starName" : "二钻"},"sort" : [152]}]}
}

没毛病

23.2.2 深度分页问题

ES是分布式的【集群】，所以会面临深度分页问题。

例如按price排序后，获取from = 990，size =10的数据：

1. 首先在每个数据分片上都排序并查询前1000条文档。

2. 然后将所有节点的结果聚合，在内存中重新排序选出前1000条文档

3. 最后从这1000条中，选取从990开始的10条文档

如果搜索页数过深，或者结果集（from + size）越大，对内存和CPU的消耗也越高。因此ES设定结果集查询的上限是10000

试试阈值

这样还没问题，+ 1

直接报错了

就是这样

【那万一真有这样的需求咋办？就没办法解决了吗？】【当然是有的】

23.2.3 深度分页解决方案

针对深度分页，ES提供了两种解决方案，官方文档：

• search after：分页时需要排序，原理是从上一次的排序值开始，查询下一页数据。官方推荐使用的方式。
• scroll：原理将排序数据形成快照，保存在内存。【内存消耗太大了】官方已经不推荐使用。

23.2.4 总结

from + size：

• 优点：支持随机翻页
• 缺点：深度分页问题，默认查询上限（from + size）是10000
• 场景：百度、京东、谷歌、淘宝这样的随机翻页搜索

after search：

• 优点：没有查询上限（单次查询的size不超过10000）
• 缺点：只能向后逐页查询，不支持随机翻页
• 场景：没有随机翻页需求的搜索，例如手机向下滚动翻页

scroll：

• 优点：没有查询上限（单次查询的size不超过10000）
• 缺点：会有额外内存消耗，并且搜索结果是非实时的
• 场景：海量数据的获取和迁移。从ES7.1开始不推荐，建议用 after search方案。