SpringBoot + Elasticsearch 电商产品搜索功能设计与实现

SpringBoot + Elasticsearch 电商产品搜索功能设计与实现

本文将详细介绍如何使用SpringBoot和Elasticsearch构建高性能的电商产品搜索系统，涵盖索引设计、核心功能实现、性能优化等关键方面。

一、系统架构设计

1.1 整体架构

电商搜索系统的典型架构如下：

┌─────────────┐    ┌──────────────┐    ┌─────────────────┐
│  MySQL数据库 │───>│  同步服务   │───>│  Elasticsearch  │
└─────────────┘    └──────────────┘    └─────────────────┘
                                           │
┌─────────────┐    ┌──────────────┐    ┌───┴─────────────┐
│  前端页面   │<──>│ SpringBoot API│<──>│ 搜索服务层     │
└─────────────┘    └──────────────┘    └─────────────────┘

1.2 技术栈选择

• Spring Boot: 2.7.x 或 3.x
• Elasticsearch: 7.17.x 或 8.x
• Spring Data Elasticsearch: 用于简化ES操作
• MySQL: 存储原始商品数据
• 消息队列: RabbitMQ/Kafka (用于数据同步)
• 分词器: IK Analyzer (中文分词)

二、索引设计

2.1 索引命名与分片策略

// 创建索引的代码示例
CreateIndexRequest request = new CreateIndexRequest("products");
request.settings(Settings.builder()
    .put("index.number_of_shards", 5)      // 主分片数
    .put("index.number_of_replicas", 2)    // 副本数
    .put("index.refresh_interval", "30s")  // 刷新间隔
    .put("analysis.analyzer.default.type", "ik_max_word")  // 默认分词器
);

分片策略建议：

• 单个分片大小控制在20GB-50GB之间
• 分片数量计算公式：总分片数 = 数据总量(GB) / 单个分片推荐大小
• 搜索密集型场景：副本数 > 主分片数，提高查询性能

2.2 商品索引映射设计

@Document(indexName = "products")
@Setting(settingPath = "es-settings.json")
public class Product {
    @Id
    private String id;
    
    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String name;  // 商品名称，支持全文搜索
    
    @Field(type = FieldType.Text, analyzer = "ik_max_word")
    private String description;  // 商品描述
    
    @Field(type = FieldType.Keyword)  // 不分词，用于精确匹配和聚合
    private String brandName;
    
    @Field(type = FieldType.Keyword)
    private String categoryName;
    
    @Field(type = FieldType.Double)
    private Double price;  // 价格，用于范围查询和排序
    
    @Field(type = FieldType.Integer)
    private Integer stock;  // 库存
    
    @Field(type = FieldType.Integer)
    private Integer sales;  // 销量
    
    @Field(type = FieldType.Object)  // 嵌套对象，存储规格信息
    private Map<String, String> specMap;
    
    @Field(type = FieldType.Boolean)
    private Boolean isOnSale;  // 是否上架
    
    @Field(type = FieldType.Date, format = DateFormat.date_time)
    private Date createTime;  // 创建时间
    
    // getter和setter方法
}

三、SpringBoot与Elasticsearch集成

3.1 添加依赖

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-elasticsearch</artifactId>
    </dependency>
    <!-- IK分词器依赖 -->
    <dependency>
        <groupId>com.github.magese</groupId>
        <artifactId>ik-analyzer</artifactId>
        <version>8.7.0</version>
    </dependency>
</dependencies>

3.2 配置Elasticsearch

spring:
  elasticsearch:
    rest:
      uris: http://localhost:9200
      # 如需认证
      # username: elastic
      # password: changeme

3.3 数据访问层

public interface ProductRepository extends ElasticsearchRepository<Product, String> {
    // 基于名称的模糊查询
    List<Product> findByNameContaining(String name);
    
    // 基于品牌的精确查询
    List<Product> findByBrandName(String brandName);
    
    // 基于价格范围的查询
    List<Product> findByPriceBetween(Double minPrice, Double maxPrice);
    
    // 分页查询
    Page<Product> findByNameContaining(String name, Pageable pageable);
}

四、核心搜索功能实现

4.1 搜索服务实现

@Service
public class ProductSearchServiceImpl implements ProductSearchService {
    
    @Autowired
    private ElasticsearchRestTemplate elasticsearchRestTemplate;
    
    @Override
    public SearchResult search(SearchRequest request) {
        // 构建查询条件
        BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
        
        // 关键词搜索
        if (StringUtils.hasText(request.getKeywords())) {
            boolQuery.must(QueryBuilders.matchQuery("name", request.getKeywords())
                    .operator(Operator.AND));
        }
        
        // 品牌过滤
        if (StringUtils.hasText(request.getBrand())) {
            boolQuery.filter(QueryBuilders.termQuery("brandName", request.getBrand()));
        }
        
        // 分类过滤
        if (StringUtils.hasText(request.getCategory())) {
            boolQuery.filter(QueryBuilders.termQuery("categoryName", request.getCategory()));
        }
        
        // 价格范围过滤
        if (request.getMinPrice() != null && request.getMaxPrice() != null) {
            boolQuery.filter(QueryBuilders.rangeQuery("price")
                    .gte(request.getMinPrice())
                    .lte(request.getMaxPrice()));
        }
        
        // 规格过滤
        if (request.getSpecs() != null && !request.getSpecs().isEmpty()) {
            for (Map.Entry<String, String> spec : request.getSpecs().entrySet()) {
                boolQuery.filter(QueryBuilders.termQuery(
                        "specMap." + spec.getKey() + ".keyword", spec.getValue()));
            }
        }
        
        // 上架状态过滤
        boolQuery.filter(QueryBuilders.termQuery("isOnSale", true));
        
        // 构建查询
        NativeSearchQueryBuilder queryBuilder = new NativeSearchQueryBuilder()
                .withQuery(boolQuery)
                .withHighlightFields(
                        new HighlightBuilder.Field("name")
                                .preTags("<em>")
                                .postTags("</em>")
                );
        
        // 排序处理
        if (StringUtils.hasText(request.getSortBy())) {
            SortOrder order = request.isAscending() ? SortOrder.ASC : SortOrder.DESC;
            queryBuilder.withSort(SortBuilders.fieldSort(request.getSortBy()).order(order));
        } else {
            // 默认按相关度排序
            queryBuilder.withSort(SortBuilders.scoreSort());
        }
        
        // 分页
        Pageable pageable = PageRequest.of(request.getPage() - 1, request.getSize());
        queryBuilder.withPageable(pageable);
        
        // 品牌聚合
        String brandAggName = "brandAgg";
        queryBuilder.addAggregation(AggregationBuilders.terms(brandAggName)
                .field("brandName").size(100));
        
        // 分类聚合
        String categoryAggName = "categoryAgg";
        queryBuilder.addAggregation(AggregationBuilders.terms(categoryAggName)
                .field("categoryName").size(100));
        
        // 规格聚合 (嵌套聚合)
        String specAggName = "specAgg";
        queryBuilder.addAggregation(AggregationBuilders.nested(specAggName, "specMap")
                .subAggregation(AggregationBuilders.terms("specKeys")
                        .field("specMap.key")
                        .subAggregation(AggregationBuilders.terms("specValues")
                                .field("specMap.value").size(50))));
        
        // 价格区间聚合
        String priceAggName = "priceAgg";
        queryBuilder.addAggregation(AggregationBuilders.range(priceAggName)
                .field("price")
                .addRange("0-100", 0, 100)
                .addRange("100-500", 100, 500)
                .addRange("500-1000", 500, 1000)
                .addRange("1000+", 1000, null));
        
        // 执行查询
        NativeSearchQuery searchQuery = queryBuilder.build();
        SearchHits<Product> searchHits = elasticsearchRestTemplate.search(searchQuery, Product.class);
        
        // 处理结果
        SearchResult result = new SearchResult();
        
        // 处理商品列表和高亮
        List<ProductDTO> products = new ArrayList<>();
        for (SearchHit<Product> hit : searchHits) {
            Product product = hit.getContent();
            ProductDTO dto = new ProductDTO();
            BeanUtils.copyProperties(product, dto);
            
            // 设置高亮
            Map<String, List<String>> highlightFields = hit.getHighlightFields();
            if (highlightFields.containsKey("name")) {
                dto.setHighlightName(highlightFields.get("name").get(0));
            }
            
            products.add(dto);
        }
        result.setProducts(products);
        
        // 处理分页信息
        result.setTotal(searchHits.getTotalHits());
        result.setPage(request.getPage());
        result.setSize(request.getSize());
        
        // 处理聚合结果
        Aggregations aggregations = searchHits.getAggregations();
        
        // 处理品牌聚合
        Terms brandTerms = aggregations.get(brandAggName);
        List<String> brands = brandTerms.getBuckets().stream()
                .map(Bucket::getKeyAsString)
                .collect(Collectors.toList());
        result.setBrands(brands);
        
        // 处理分类聚合
        Terms categoryTerms = aggregations.get(categoryAggName);
        List<String> categories = categoryTerms.getBuckets().stream()
                .map(Bucket::getKeyAsString)
                .collect(Collectors.toList());
        result.setCategories(categories);
        
        // 处理价格区间聚合
        Range priceRange = aggregations.get(priceAggName);
        List<Range.Bucket> priceBuckets = priceRange.getBuckets();
        // 构建价格区间结果...
        
        // 处理规格聚合
        // 解析嵌套聚合获取规格键值对...
        
        return result;
    }
}

4.2 控制器实现

@RestController
@RequestMapping("/api/search")
public class SearchController {
    
    @Autowired
    private ProductSearchService productSearchService;
    
    @GetMapping
    public ResponseEntity<SearchResult> search(
            @RequestParam(required = false) String keywords,
            @RequestParam(required = false) String brand,
            @RequestParam(required = false) String category,
            @RequestParam(required = false) Double minPrice,
            @RequestParam(required = false) Double maxPrice,
            @RequestParam(required = false) String sortBy,
            @RequestParam(defaultValue = "false") Boolean ascending,
            @RequestParam(defaultValue = "1") Integer page,
            @RequestParam(defaultValue = "20") Integer size,
            @RequestParam(required = false) MultiValueMap<String, String> specs) {
        
        // 构建搜索请求
        SearchRequest request = new SearchRequest();
        request.setKeywords(keywords);
        request.setBrand(brand);
        request.setCategory(category);
        request.setMinPrice(minPrice);
        request.setMaxPrice(maxPrice);
        request.setSortBy(sortBy);
        request.setAscending(ascending);
        request.setPage(page);
        request.setSize(size);
        
        // 处理规格参数 (以spec_开头的参数)
        if (specs != null) {
            Map<String, String> specMap = new HashMap<>();
            for (Map.Entry<String, List<String>> entry : specs.entrySet()) {
                String key = entry.getKey();
                if (key.startsWith("spec_")) {
                    String specName = key.substring(5); // 去掉spec_前缀
                    specMap.put(specName, entry.getValue().get(0));
                }
            }
            request.setSpecs(specMap);
        }
        
        // 执行搜索
        SearchResult result = productSearchService.search(request);
        return ResponseEntity.ok(result);
    }
}

五、数据同步策略

5.1 基于Canal的数据同步

使用Canal监听MySQL Binlog，实时同步数据到Elasticsearch：

@Component
public class ProductSyncHandler {
    
    @Autowired
    private ElasticsearchRestTemplate elasticsearchRestTemplate;
    
    public void handleInsert(Product product) {
        elasticsearchRestTemplate.save(product);
    }
    
    public void handleUpdate(Product product) {
        elasticsearchRestTemplate.save(product);
    }
    
    public void handleDelete(String productId) {
        elasticsearchRestTemplate.delete(productId, Product.class);
    }
}

5.2 基于消息队列的数据同步

@Component
public class ProductMessageListener {
    
    @Autowired
    private ElasticsearchRestTemplate elasticsearchRestTemplate;
    
    @RabbitListener(queues = "product-index-queue")
    public void handleProductMessage(ProductMessage message) {
        switch (message.getAction()) {
            case INSERT:
                elasticsearchRestTemplate.save(message.getProduct());
                break;
            case UPDATE:
                elasticsearchRestTemplate.save(message.getProduct());
                break;
            case DELETE:
                elasticsearchRestTemplate.delete(message.getProductId(), Product.class);
                break;
        }
    }
}

六、高级功能实现

6.1 相关性优化

// 使用function_score提升搜索结果相关性
FunctionScoreQueryBuilder functionScoreQuery = QueryBuilders.functionScoreQuery(
        boolQuery,
        new FunctionScoreQueryBuilder.FilterFunctionBuilder[] {
                new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                        QueryBuilders.termQuery("isHot", true),
                        ScoreFunctionBuilders.weightFactorFunction(1.5f)),
                new FunctionScoreQueryBuilder.FilterFunctionBuilder(
                        QueryBuilders.rangeQuery("sales").gt(1000),
                        ScoreFunctionBuilders.weightFactorFunction(1.2f))
        }
);

queryBuilder.withQuery(functionScoreQuery);

6.2 同义词支持

在elasticsearch.yml中配置同义词文件：

index:
  analysis:
    filter:
      synonym_filter:
        type: synonym
        synonyms_path: config/synonyms.txt
    analyzer:
      ik_syno:
        tokenizer: ik_max_word
        filter:
          - synonym_filter

synonyms.txt 文件内容示例：

手机,智能手机,移动端
电脑,笔记本电脑,台式机

6.3 拼音搜索

// 添加拼音分词器配置
@Setting(settingPath = "es-settings.json")
public class Product {
    // 使用多字段实现拼音搜索
    @MultiField(mainField = @Field(type = FieldType.Text, analyzer = "ik_max_word"),
               otherFields = { @InnerField(suffix = "pinyin", type = FieldType.Text, analyzer = "pinyin_analyzer") })
    private String name;
    // ...
}

七、性能优化策略

7.1 索引层面优化

• 合理设置分片和副本：根据数据量和查询压力调整
• 禁用不必要的字段索引：对不需要搜索的字段设置index: false
• 使用适当的字段类型：keyword vs text，数值类型等
• 使用延迟加载：非必要字段使用enabled: false
• 索引生命周期管理：定期清理历史数据

7.2 查询层面优化

• 避免使用wildcard查询：特别是前缀通配符
• 使用filter代替query：filter结果可以缓存，提高性能
• 分页深度限制：避免使用from/size进行深度分页，使用scroll或search_after
• 字段选择性加载：使用_source过滤只返回需要的字段
• 批量操作：使用bulk API进行批量写入和查询

// 优化示例：使用source filtering只返回必要字段
queryBuilder.withSourceFilter(new FetchSourceFilter(
    new String[] {"id", "name", "price", "brandName"}, null
));

7.3 缓存优化

• Redis缓存热门搜索结果
• 使用Elasticsearch的filter缓存
• 实现搜索结果缓存

@Service
public class CachedSearchService {
    
    @Autowired
    private RedisTemplate<String, SearchResult> redisTemplate;
    
    @Autowired
    private ProductSearchService productSearchService;
    
    public SearchResult search(SearchRequest request) {
        // 生成缓存key
        String cacheKey = generateCacheKey(request);
        
        // 尝试从缓存获取
        SearchResult cachedResult = redisTemplate.opsForValue().get(cacheKey);
        if (cachedResult != null) {
            return cachedResult;
        }
        
        // 执行搜索
        SearchResult result = productSearchService.search(request);
        
        // 存入缓存
        redisTemplate.opsForValue().set(cacheKey, result, 5, TimeUnit.MINUTES);
        
        return result;
    }
    
    private String generateCacheKey(SearchRequest request) {
        // 生成唯一缓存key
        return "search:" + DigestUtils.md5Hex(JSON.toJSONString(request));
    }
}

八、部署与监控

8.1 Elasticsearch集群部署

推荐至少3节点集群，配置分离的主节点、数据节点和协调节点：

# elasticsearch.yml 配置示例
node.name: node-1
cluster.name: es-product-cluster
path.data: /data/es/data
path.logs: /data/es/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
discovery.seed_hosts: ["node-1", "node-2", "node-3"]
cluster.initial_master_nodes: ["node-1"]

8.2 监控与告警

• 使用Kibana监控ES集群状态
• 配置JVM监控
• 设置索引健康检查告警
• 监控查询性能

九、总结

通过合理的索引设计、优化的查询策略和可靠的数据同步机制，可以构建高性能、可扩展的电商产品搜索系统。本文介绍的方案涵盖了从基础架构到高级功能的各个方面，适用于构建生产级别的电商搜索服务。

在实际应用中，还需要根据业务场景和数据规模进行进一步的调优和优化，不断提升用户搜索体验。