Elastic Stack 日志分析(四)- Elasticsearch Java API 操作详解
|字数总计:3.8k|阅读时长:19分钟|阅读量:
1. 创建Maven项目
我们在 IDEA 开发工具中创建 Maven 项目(模块也可)ESTest
。
修改pom
文件,添加 Maven 依赖关系:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
| <dependencies> <dependency> <groupId>org.elasticsearch</groupId> <artifactId>elasticsearch</artifactId> <version>7.17.0</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>7.8.0</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.17.1</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.17.1</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.10.4</version> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> </dependency> </dependencies>
|
2. 客户端对象
创建com.elk.es.test
类,代码中创建 Elasticsearch 客户端对象,这里我们采用高级 REST 客户端对象。事实上,在 7.15.0 版本中,ES 已经弃用了 Java REST Client,使用 Java API Client,但是目前市场都还用不到那么新的版本,所以这里还是使用高级 REST API。
如果有感兴趣的,可以去官网了解最新的 Java API Client:https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/7.17/connecting.html#connecting
1 2 3 4 5 6 7 8 9
| RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
......
client.close();
|
注意:9200
端口为 Elasticsearch 的 Web 通信端口,localhost
为启动 ES 服务的主机名。
3. 索引操作
ES 服务器正常启动后,可以通过 Java API 客户端对象对 ES 索引进行操作。
3.1 创建索引
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
CreateIndexRequest request = new CreateIndexRequest("user"); CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT); boolean acknowledged = response.isAcknowledged(); System.out.println("操作状态 = " + acknowledged); client.close(); }
|
操作结果:
3.2 查看索引
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
GetIndexRequest request = new GetIndexRequest("user"); GetIndexResponse response = client.indices().get(request,RequestOptions.DEFAULT); System.out.println("aliases:"+response.getAliases()); System.out.println("mappings:"+response.getMappings()); System.out.println("settings:"+response.getSettings()); client.close(); }
|
操作结果:
3.3 删除索引
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
DeleteIndexRequest request = new DeleteIndexRequest("user"); AcknowledgedResponse response = client.indices().delete(request,RequestOptions.DEFAULT); System.out.println("操作结果 : " + response.isAcknowledged()); client.close(); }
|
操作结果:
4. 文档操作
4.1 添加文档
新增文档创建数据模型:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
| class User { private String name; private Integer age; private String sex;
public String getName() { return name; } public void setName(String name) { this.name = name; } public Integer getAge() { return age; } public void setAge(Integer age) { this.age = age; } public String getSex() { return sex; } public void setSex(String sex) { this.sex = sex; } }
|
创建数据,添加到文档中:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
IndexRequest request = new IndexRequest(); request.index("user").id("1001"); User user = new User(); user.setName("zhangsan"); user.setAge(30); user.setSex("男"); ObjectMapper objectMapper = new ObjectMapper(); String productJson = objectMapper.writeValueAsString(user); request.source(productJson, XContentType.JSON); IndexResponse response = client.index(request, RequestOptions.DEFAULT); System.out.println("_index:" + response.getIndex()); System.out.println("_id:" + response.getId()); System.out.println("_result:" + response.getResult()); client.close(); }
|
操作结果:
4.1 修改文档
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
UpdateRequest request = new UpdateRequest(); request.index("user").id("1001"); request.doc(XContentType.JSON, "sex", "女"); UpdateResponse response = client.update(request, RequestOptions.DEFAULT); System.out.println("_index:" + response.getIndex()); System.out.println("_id:" + response.getId()); System.out.println("_result:" + response.getResult()); client.close(); }
|
操作结果:
4.3 查询文档
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
GetRequest request = new GetRequest().index("user").id("1001"); GetResponse response = client.get(request, RequestOptions.DEFAULT); System.out.println("_index:" + response.getIndex()); System.out.println("_type:" + response.getType()); System.out.println("_id:" + response.getId()); System.out.println("source:" + response.getSourceAsString()); client.close(); }
|
操作结果:
4.4 删除文档
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
DeleteRequest request = new DeleteRequest().index("user").id("1"); DeleteResponse response = client.delete(request, RequestOptions.DEFAULT); System.out.println(response.toString()); client.close(); }
|
操作结果:
4.5 批量操作
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
BulkRequest request = new BulkRequest(); request.add(new IndexRequest().index("user").id("1001").source(XContentType.JSON, "name", "zhangsan")); request.add(new IndexRequest().index("user").id("1002").source(XContentType.JSON, "name", "lisi")); request.add(new IndexRequest().index("user").id("1003").source(XContentType.JSON, "name", "wangwu")); BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT); System.out.println("took:" + responses.getTook()); System.out.println("items:" + responses.getItems()); client.close(); }
|
操作结果:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
BulkRequest request = new BulkRequest(); request.add(new DeleteRequest().index("user").id("1001")); request.add(new DeleteRequest().index("user").id("1002")); request.add(new DeleteRequest().index("user").id("1003")); BulkResponse responses = client.bulk(request, RequestOptions.DEFAULT); System.out.println("took:" + responses.getTook()); System.out.println("items:" + responses.getItems()); client.close(); }
|
操作结果:
5. 高级查询
5.1 请求体查询
5.1.1 查询所有索引数据
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.2 term 查询,查询条件为关键字
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.termQuery("age", "30")); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.3 分页查询
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); sourceBuilder.from(0); sourceBuilder.size(2); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.4 数据排序
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); sourceBuilder.sort("age", SortOrder.ASC); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.5 过滤字段
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.matchAllQuery()); String[] excludes = {}; String[] includes = {"name", "age"}; sourceBuilder.fetchSource(includes, excludes); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.6 Bool 查询
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery(); boolQueryBuilder.must(QueryBuilders.matchQuery("age", "30")); boolQueryBuilder.mustNot(QueryBuilders.matchQuery("name", "zhangsan")); boolQueryBuilder.should(QueryBuilders.matchQuery("sex", "男")); sourceBuilder.query(boolQueryBuilder); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.7 范围查询
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("age"); rangeQuery.gte("30"); rangeQuery.lte("40"); sourceBuilder.query(rangeQuery); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.1.8 模糊查询
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest(); request.indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.query(QueryBuilders.fuzzyQuery("name","zhangsan").fuzziness(Fuzziness.ONE)); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took:" + response.getTook()); System.out.println("timeout:" + response.isTimedOut()); System.out.println("total:" + hits.getTotalHits()); System.out.println("MaxScore:" + hits.getMaxScore()); System.out.println("hits========>>"); for (SearchHit hit : hits) { System.out.println(hit.getSourceAsString()); } System.out.println("<<========"); client.close(); }
|
操作结果:
5.2 高亮查询
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest().indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("name","zhangsan"); sourceBuilder.query(termsQueryBuilder); HighlightBuilder highlightBuilder = new HighlightBuilder(); highlightBuilder.preTags("<font color='red'>"); highlightBuilder.postTags("</font>"); highlightBuilder.field("name"); sourceBuilder.highlighter(highlightBuilder); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println("took::"+response.getTook()); System.out.println("time_out::"+response.isTimedOut()); System.out.println("total::"+hits.getTotalHits()); System.out.println("max_score::"+hits.getMaxScore()); System.out.println("hits::::>>"); for (SearchHit hit : hits) { String sourceAsString = hit.getSourceAsString(); System.out.println(sourceAsString); Map<String, HighlightField> highlightFields = hit.getHighlightFields(); System.out.println(highlightFields); } System.out.println("<<::::"); client.close(); }
|
操作结果:
5.3 聚合查询
5.3.1 最大年龄
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest().indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.max("maxAge").field("age")); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println(response); client.close(); }
|
操作结果:
5.3.2 分组统计
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
| public static void main(String[] args) throws IOException { RestHighLevelClient client = new RestHighLevelClient( RestClient.builder(new HttpHost("localhost", 9200, "http")) );
SearchRequest request = new SearchRequest().indices("student"); SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); sourceBuilder.aggregation(AggregationBuilders.terms("age_groupby").field("age")); request.source(sourceBuilder); SearchResponse response = client.search(request, RequestOptions.DEFAULT); SearchHits hits = response.getHits(); System.out.println(response); client.close(); }
|
操作结果: