在Docker中使用CONNECT SE访问远程JSON数据的演示示例

107 阅读4分钟

CONNECT是一个存储引擎(SE)插件,用于访问外部、本地或远程数据。在这篇博客中,我们将展示如何在Docker容器中安装CONNECT存储引擎以及如何在容器之间共享JSON数据。

在Docker中启用CONNECT SE插件

CONNECT SE需要安装在容器内,以便使用它。要了解如何做到这一点,请查看在MariaDB Docker Library容器中安装插件

在远程服务器上创建JSON数据

CONNECT SE到MariaDB的最重要的特点是可以灵活地从各种数据源创建表,比如同一个数据库和其他DMBS的表或不同格式的文件。你可以在CONNECT表格类型中阅读更多内容。这里我们将使用CONNECT JSON表类型

我们将有2个容器,mariadb-remote ,作为源数据所在的服务器,以及mariadb-local ,我们将把它作为一个本地服务器,我们将从源服务器创建一个表。所以我们首先在源服务器上创建JSON数据。

  • 为了有一个远程连接而创建网络
$ docker network create mynetwork
  • 使用镜像my_connect_img 并通过为容器启动时创建的用户指定权限来启动源容器mariadb-remote ,你会在之前的博客中发现。
$ docker run --rm -d --name mariadb-remote -v$PWD/local_init_dir1:/docker-entrypoint-initdb.d \
  -e MARIADB_USER=anel  -e MARIADB_PASSWORD=anel -e MARIADB_DATABASE=web_users \
  --network mynetwork -e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 my_connect_img
  • 验证用户的授权
$ docker exec -it mariadb-remote mariadb -uanel -panel web_users  -e "show grants for current_user;"
+--------------------------------------------------------------------------------------------------------------+
| Grants for anel@%                                                                                            |
+--------------------------------------------------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO `anel`@`%` IDENTIFIED BY PASSWORD '*1F81CD5C3293B40FA2B98C69E8495874BE999E34' |
| GRANT ALL PRIVILEGES ON `web\_users`.* TO `anel`@`%`                                                         |
+--------------------------------------------------------------------------------------------------------------+
  • 由于我们是从安装了CONNECT的镜像中构建的,所以不需要检查SE的存在。
  • 创建JSON数据。

我们将使用来自users1users2的数据,通过SQL语句将它们存储在一个文件中,并使用表发现功能从该文件中创建CONNECT表。其步骤是。

- 启动客户端

docker exec -it mariadb-remote mariadb -uanel -panel web_users

- 执行查询,用set语句创建文件,并选择进入outfile

set @web_users='[{  "id": 1,  "name": "Leanne Graham",  "username": "Bret",  "email": "Sincere@april.biz",  "address": {    "street": "Kulas Light",    "suite": "Apt. 556",    "city": "Gwenborough",    "zipcode": "92998-3874",    "geo": {      "lat": "-37.3159",      "lng": "81.1496"    }  },  "phone": "1-770-736-8031 x56442",  "website": "hildegard.org",  "company": {    "name": "Romaguera-Crona",    "catchPhrase": "Multi-layered client-server neural-net",    "bs": "harness real-time e-markets"  }},{  "id": 2,  "name": "Ervin Howell",  "username": "Antonette",  "email": "Shanna@melissa.tv",  "address": {    "street": "Victor Plains",    "suite": "Suite 879",    "city": "Wisokyburgh",    "zipcode": "90566-7771",    "geo": {      "lat": "-43.9509",      "lng": "-34.4618"    }  },  "phone": "010-692-6593 x09125",  "website": "anastasia.net",  "company": {    "name": "Deckow-Crist",    "catchPhrase": "Proactive didactic contingency",    "bs": "synergize scalable supply-chains"  }}]';
SET STATEMENT SQL_MODE= CONCAT(@@SQL_MODE, ',NO_BACKSLASH_ESCAPES') FOR SELECT @web_users into outfile 'web_users.json';

- 从一个文件中创建CONNECT表并查看结果

CREATE TABLE webusers
ENGINE=CONNECT TABLE_TYPE=JSON FILE_NAME='web_users.json';

MariaDB [web_users]> show create table webusers\G
*************************** 1. row ***************************
       Table: webusers
Create Table: CREATE TABLE `webusers` (
  `id` int(1) NOT NULL `JPATH`='$.id',
  `name` char(13) NOT NULL `JPATH`='$.name',
  `username` char(9) NOT NULL `JPATH`='$.username',
  `email` char(17) NOT NULL `JPATH`='$.email',
  `address_street` char(13) NOT NULL `JPATH`='$.address.street',
  `address_suite` char(9) NOT NULL `JPATH`='$.address.suite',
  `address_city` char(11) NOT NULL `JPATH`='$.address.city',
  `address_zipcode` char(10) NOT NULL `JPATH`='$.address.zipcode',
  `address_geo_lat` char(8) NOT NULL `JPATH`='$.address.geo.lat',
  `address_geo_lng` char(8) NOT NULL `JPATH`='$.address.geo.lng',
  `phone` char(21) NOT NULL `JPATH`='$.phone',
  `website` char(13) NOT NULL `JPATH`='$.website',
  `company_name` char(15) NOT NULL `JPATH`='$.company.name',
  `company_catchPhrase` char(38) NOT NULL `JPATH`='$.company.catchPhrase',
  `company_bs` char(32) NOT NULL `JPATH`='$.company.bs'
) ENGINE=CONNECT DEFAULT CHARSET=utf8mb4 `TABLE_TYPE`='JSON' `FILE_NAME`='web_users.json'
1 row in set (0.000 sec)

从远程服务器创建本地表

同样,我们需要创建名为mariadb-local 的本地容器/服务器。以下是需要做的步骤。

启动本地容器mariadb-local (我们再次在一个文件中定义授予,并使用卷来应用它们)。

请按照之前的步骤安装该插件,为你的用户验证授权,重启服务器并检查该插件是否工作。

$ docker run --rm -d --name mariadb-local \
-e MARIADB_USER=local_user  -e MARIADB_PASSWORD=local_user_pw -e MARIADB_DATABASE=test \
--network mynetwork -v$PWD/local_init_dir:/docker-entrypoint-initdb.d \
-e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 my_connect_img
  • 使用远程连接到源服务器创建表

- 启动客户端

$ docker exec -it mariadb-local mariadb -ulocal_user -plocal_user_pw test

- 从远程服务器创建表(注意,本地用户至少需要有FILE权限才能用于向外连接的表)。

MariaDB [test]> show tables;
Empty set (0.000 sec)

MariaDB [test]> CREATE TABLE local_web_user
                ENGINE = CONNECT
                TABLE_TYPE = MYSQL
                DBNAME = 'web_users'
                TABNAME = 'webusers'
                CONNECTION = 'mysql://anel:anel@mariadb-remote/';
Query OK, 0 rows affected (0.041 sec)

MariaDB [test]> show tables;
+----------------+
| Tables_in_test |
+----------------+
| local_web_user |
+----------------+
1 row in set (0.000 sec)

瞧,我们现在有了来自远程服务器的数据。

  • 创建新表

现在我们可以创建InnoDB表,如果我们愿意的话。

MariaDB [test]> create table new_table as select name, email, address_street from local_web_user;
MariaDB [test]> show create table new_table;
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table     | Create Table                                                                                                                                                     |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| new_table | CREATE TABLE `new_table` (
  `name` char(13) NOT NULL,
  `email` char(17) NOT NULL,
  `address_street` char(13) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 |
+-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.000 sec)

MariaDB [test]> select * from new_table;
+---------------+-------------------+----------------+
| name          | email             | address_street |
+---------------+-------------------+----------------+
| Leanne Graham | Sincere@april.biz | Kulas Light    |
| Ervin Howell  | Shanna@melissa.tv | Victor Plains  |
+---------------+-------------------+----------------+
2 rows in set (0.000 sec)

- 人们可以使用一个特定的查询,以便从远程创建表。

MariaDB [test]> CREATE TABLE local_web_user1 ENGINE = CONNECT TABLE_TYPE = MYSQL SRCDEF = 'SELECT * FROM web_users.webusers WHERE id=1' CONNECTION = 'mysql://anel:anel@mariadb-remote/';

其结果是。

MariaDB [test]> select * from local_web_user1;
+----+---------------+----------+-------------------+----------------+---------------+--------------+-----------------+-----------------+-----------------+-----------------------+---------------+-----------------+----------------------------------------+-----------------------------+
| id | name          | username | email             | address_street | address_suite | address_city | address_zipcode | address_geo_lat | address_geo_lng | phone                 | website       | company_name    | company_catchPhrase                    | company_bs                  |
+----+---------------+----------+-------------------+----------------+---------------+--------------+-----------------+-----------------+-----------------+-----------------------+---------------+-----------------+----------------------------------------+-----------------------------+
|  1 | Leanne Graham | Bret     | Sincere@april.biz | Kulas Light    | Apt. 556      | Gwenborough  | 92998-3874      | -37.3159        | 81.1496         | 1-770-736-8031 x56442 | hildegard.org | Romaguera-Crona | Multi-layered client-server neural-net | harness real-time e-markets |
+----+---------------+----------+-------------------+----------------+---------------+--------------+-----------------+-----------------+-----------------+-----------------------+---------------+-----------------+----------------------------------------+-----------------------------+
1 row in set (0.001 sec)

结论和未来工作

这篇博客的主要目的是用简单的语法验证CONNECT SE的功能,从自定义数据源创建数据,并从远程服务器获取数据。

这还不是全部。我们只是看到了一个单一的用例,并触及了这个伟大的存储引擎的表面。