Dubbo(98)如何在人工智能系统中应用Dubbo?

78 阅读5分钟

在人工智能系统中应用Dubbo,可以通过以下步骤实现:

  1. 项目结构设计:规划项目的模块结构,通常包括服务接口模块、各个服务提供者模块(如模型训练服务、模型推理服务、数据预处理服务等)、服务消费者模块(如前端应用、后台管理系统等)。
  2. 服务接口定义:定义各个服务的接口,确保服务提供者和消费者能够共享相同的接口。
  3. 服务提供者实现:实现各个服务接口,并配置Dubbo提供服务。
  4. 服务消费者调用:在服务消费者中引用服务接口,并通过Dubbo调用远程服务。
  5. 服务注册与发现:配置注册中心(如Zookeeper)以实现服务注册与发现。
  6. 配置管理:管理项目的配置文件,确保服务提供者和消费者能够正确连接到注册中心并发现彼此。
  7. 测试与部署:测试服务的调用,确保服务能够正常工作,并将服务部署到生产环境。

以下是一个详细的示例,展示如何在人工智能系统中应用Dubbo。

1. 项目结构设计

我们将创建一个包含多个模块的项目:dubbo-apimodel-training-servicemodel-inference-servicedata-preprocessing-servicefrontend-application

ai-system
├── dubbo-api
│   └── src/main/java/com/example/dubbo/api
│       ├── ModelTrainingService.java
│       ├── ModelInferenceService.java
│       └── DataPreprocessingService.java
├── model-training-service
│   └── src/main/java/com/example/modeltraining
│       ├── service
│       │   └── ModelTrainingServiceImpl.java
│       ├── config
│       │   └── DubboProviderConfig.java
│       └── ModelTrainingServiceApplication.java
├── model-inference-service
│   └── src/main/java/com/example/modelinference
│       ├── service
│       │   └── ModelInferenceServiceImpl.java
│       ├── config
│       │   └── DubboProviderConfig.java
│       └── ModelInferenceServiceApplication.java
├── data-preprocessing-service
│   └── src/main/java/com/example/datapreprocessing
│       ├── service
│       │   └── DataPreprocessingServiceImpl.java
│       ├── config
│       │   └── DubboProviderConfig.java
│       └── DataPreprocessingServiceApplication.java
├── frontend-application
│   └── src/main/java/com/example/frontend
│       ├── controller
│       │   └── AIController.java
│       ├── config
│       │   └── DubboConsumerConfig.java
│       └── FrontendApplication.java
└── pom.xml

2. 服务接口定义

2.1 创建 dubbo-api 模块

创建 dubbo-api 模块的 pom.xml 文件:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/POM/4.0.0 http://www.w3.org/2001/04/xmldsig-more#">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>dubbo-api</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo</artifactId>
            <version>2.7.8</version>
        </dependency>
    </dependencies>
</project>

2.2 定义服务接口

dubbo-api/src/main/java/com/example/dubbo/api 目录下创建 ModelTrainingServiceModelInferenceServiceDataPreprocessingService 接口:

package com.example.dubbo.api;

public interface ModelTrainingService {
    String trainModel(String data);
}

public interface ModelInferenceService {
    String inferModel(String data);
}

public interface DataPreprocessingService {
    String preprocessData(String data);
}

3. 服务提供者实现

3.1 创建 model-training-service 模块

创建 model-training-service 模块的 pom.xml 文件:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/POM/4.0.0 http://www.w3.org/2001/04/xmldsig-more#">

    <parent>
        <groupId>com.example</groupId>
        <artifactId>ai-system</artifactId>
        <version>1.0-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <artifactId>model-training-service</artifactId>

    <dependencies>
        <dependency>
            <groupId>com.example</groupId>
            <artifactId>dubbo-api</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
            <version>2.7.8</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
        </dependency>
    </dependencies>
</project>

3.2 实现模型训练服务接口

model-training-service/src/main/java/com/example/modeltraining/service 目录下创建 ModelTrainingServiceImpl 类:

package com.example.modeltraining.service;

import com.example.dubbo.api.ModelTrainingService;
import org.apache.dubbo.config.annotation.DubboService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@DubboService
public class ModelTrainingServiceImpl implements ModelTrainingService {
    private static final Logger logger = LoggerFactory.getLogger(ModelTrainingServiceImpl.class);

    @Override
    public String trainModel(String data) {
        logger.info("Training model with data: {}", data);
        // 模型训练逻辑
        return "Model trained with data: " + data;
    }
}

3.3 配置Dubbo服务

model-training-service/src/main/java/com/example/modeltraining/config 目录下创建 DubboProviderConfig 类:

package com.example.modeltraining.config;

import org.apache.dubbo.config.spring.context.annotation.EnableDubbo;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableDubbo(scanBasePackages = "com.example.modeltraining.service")
public class DubboProviderConfig {
}

3.4 创建启动类

model-training-service/src/main/java/com/example/modeltraining 目录下创建 ModelTrainingServiceApplication 类:

package com.example.modeltraining;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class ModelTrainingServiceApplication {
    public static void main(String[] args) {
        SpringApplication.run(ModelTrainingServiceApplication.class, args);
    }
}

3.5 配置文件

model-training-service/src/main/resources 目录下创建 application.yml 配置文件:

spring:
  application:
    name: model-training-service
  main:
    web-application-type: none

dubbo:
  application:
    name: model-training-service
  registry:
    address: zookeeper://localhost:2181
  protocol:
    name: dubbo
    port: 20880
  scan:
    base-packages: com.example.modeltraining.service

logging:
  level:
    com.example.modeltraining: INFO
  file:
    name: logs/model-training-service.log

3.6 创建 model-inference-service 模块

创建 model-inference-service 模块的 pom.xml 文件,类似于 model-training-service 模块。

3.7 实现模型推理服务接口

model-inference-service/src/main/java/com/example/modelinference/service 目录下创建 ModelInferenceServiceImpl 类:

package com.example.modelinference.service;

import com.example.dubbo.api.ModelInferenceService;
import org.apache.dubbo.config.annotation.DubboService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@DubboService
public class ModelInferenceServiceImpl implements ModelInferenceService {
    private static final Logger logger = LoggerFactory.getLogger(ModelInferenceServiceImpl.class);

    @Override
    public String inferModel(String data) {
        logger.info("Inferring model with data: {}", data);
        // 模型推理逻辑
        return "Inference result for data: " + data;
    }
}

3.8 配置Dubbo服务和启动类

model-inference-service 模块中配置Dubbo服务和启动类,类似于 model-training-service 模块。

3.9 创建 data-preprocessing-service 模块

创建 data-preprocessing-service 模块的 pom.xml 文件,类似于 model-training-service 模块。

3.10 实现数据预处理服务接口

data-preprocessing-service/src/main/java/com/example/datapreprocessing/service 目录下创建 DataPreprocessingServiceImpl 类:

package com.example.datapreprocessing.service;

import com.example.dubbo.api.DataPreprocessingService;
import org.apache.dubbo.config.annotation.DubboService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@DubboService
public class DataPreprocessingServiceImpl implements DataPreprocessingService {
    private static final Logger logger = LoggerFactory.getLogger(DataPreprocessingServiceImpl.class);

    @Override
    public String preprocessData(String data) {
        logger.info("Preprocessing data: {}", data);
        // 数据预处理逻辑
        return "Preprocessed data: " + data;
    }
}

3.11 配置Dubbo服务和启动类

data-preprocessing-service 模块中配置Dubbo服务和启动类,类似于 model-training-service 模块。

4. 服务消费者调用

4.1 创建 frontend-application 模块

创建 frontend-application 模块的 pom.xml 文件:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/POM/4.0.0 http://www.w3.org/2001/04/xmldsig-more#">

    <parent>
        <groupId>com.example</groupId>
        <artifactId>ai-system</artifactId>
        <version>1.0-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <artifactId>frontend-application</artifactId>

    <dependencies>
        <dependency>
            <groupId>com.example</groupId>
            <artifactId>dubbo-api</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.dubbo</groupId>
            <artifactId>dubbo-spring-boot-starter</artifactId>
            <version>2.7.8</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
        </dependency>
    </dependencies>
</project>

4.2 创建控制器

frontend-application/src/main/java/com/example/frontend/controller 目录下创建 AIController 类:

package com.example.frontend.controller;

import com.example.dubbo.api.DataPreprocessingService;
import com.example.dubbo.api.ModelInferenceService;
import com.example.dubbo.api.ModelTrainingService;
import org.apache.dubbo.config.annotation.DubboReference;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class AIController {
    private static final Logger logger = LoggerFactory.getLogger(AIController.class);

    @DubboReference
    private DataPreprocessingService dataPreprocessingService;

    @DubboReference
    private ModelTrainingService modelTrainingService;

    @DubboReference
    private ModelInferenceService modelInferenceService;

    @GetMapping("/preprocessData")
    public String preprocessData(@RequestParam String data) {
        logger.info("Preprocessing data: {}", data);
        return dataPreprocessingService.preprocessData(data);
    }

    @GetMapping("/trainModel")
    public String trainModel(@RequestParam String data) {
        logger.info("Training model with data: {}", data);
        return modelTrainingService.trainModel(data);
    }

    @GetMapping("/inferModel")
    public String inferModel(@RequestParam String data) {
        logger.info("Inferring model with data: {}", data);
        return modelInferenceService.inferModel(data);
    }
}

4.3 配置Dubbo消费

frontend-application/src/main/java/com/example/frontend/config 目录下创建 DubboConsumerConfig 类:

package com.example.frontend.config;

import org.apache.dubbo.config.spring.context.annotation.EnableDubbo;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableDubbo(scanBasePackages = "com.example.frontend.controller")
public class DubboConsumerConfig {
}

4.4 创建启动类

frontend-application/src/main/java/com/example/frontend 目录下创建 FrontendApplication 类:

package com.example.frontend;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class FrontendApplication {
    public static void main(String[] args) {
        SpringApplication.run(FrontendApplication.class, args);
    }
}

4.5 配置文件

frontend-application/src/main/resources 目录下创建 application.yml 配置文件:

spring:
  application:
    name: frontend-application

dubbo:
  application:
    name: frontend-application
  registry:
    address: zookeeper://localhost:2181
  protocol:
    name: dubbo
  scan:
    base-packages: com.example.frontend.controller

logging:
  level:
    com.example.frontend: INFO
  file:
    name: logs/frontend-application.log

5. 根项目的 pom.xml

在根项目 ai-system 中创建 pom.xml 文件,定义模块和依赖管理:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.w3.org/POM/4.0.0 http://www.w3.org/2001/04/xmldsig-more#">

    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>ai-system</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <modules>
        <module>dubbo-api</module>
        <module>model-training-service</module>
        <module>model-inference-service</module>
        <module>data-preprocessing-service</module>
        <module>frontend-application</module>
    </modules>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.apache.dubbo</groupId>
                <artifactId>dubbo</artifactId>
                <version>2.7.8</version>
            </dependency>
            <dependency>
                <groupId>org.apache.dubbo</groupId>
                <artifactId>dubbo-spring-boot-starter</artifactId>
                <version>2.7.8</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.8.1</version>
                    <configuration>
                        <source>1.8</source>
                        <target>1.8</target>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <version>2.3.4.RELEASE</version>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

6. 启动Zookeeper

确保Zookeeper在本地运行,默认端口为 2181。可以通过下载Zookeeper并运行以下命令启动Zookeeper:

bin/zkServer.sh start

7. 启动服务提供者和消费者

  1. 启动模型训练服务:运行 ModelTrainingServiceApplication 类。
  2. 启动模型推理服务:运行 ModelInferenceServiceApplication 类。
  3. 启动数据预处理服务:运行 DataPreprocessingServiceApplication 类。
  4. 启动前端应用:运行 FrontendApplication 类。

8. 测试服务

访问前端应用的数据预处理、模型训练和模型推理接口:

curl http://localhost:8080/preprocessData?data=sampleData
curl http://localhost:8080/trainModel?data=sampleData
curl http://localhost:8080/inferModel?data=sampleData