WL#14254: Extended test suite with Docker environment

There is an extended set of tests distributed as part of the project
which are never executed within a regular test run. This is because
they require multiple MySQL server instances, or instances with a
specific configuration to be available.

Up until now, the testing infrastructure was optimized for a scenario
where it assumed that a MySQL server instance is available either in the
local system or on some other network-accessible host.

With the introduction of the Docker-based test setup in WL#15353, it is
now easier to address these limitations and migrate to a fully
container-based testing environment on top of something like Docker's
compose, where a testing container can communicate with other containers
that act as MySQL server instances and other service discovery
components enabling automated end-to-end tests for existing client
features such as Unix sockets, IPv6, TLS certificates, DNS SRV and
multi-host connections alongside effective regression and compatibility
testing with older MySQL versions.

This patch extends the existing Docker-based test setup to account for
all these scenarios, ensuring the full test suite can run in a
consistent and replicable setting.

Apart from the testing container, the setup now includes multiple
containers running MySQL server instances with different configurations.
Users are still allowed to run the default set of tests using an
instance available at a specific host. Otherwise, the tests run using
an instance on one of the containers, which is also used as a fallback
for tests using Unix sockets, on macOS, where it is not possible to
connect from the container to the host using that socket, or on Linux,
when the MYSQLX_SOCKET environment variable is not specified.

Change-Id: Ic7b4f524474ce527d047acff3ca8080bfbbc3f40

Author: ruiquelhas

CONTRIBUTING.md

@@ -81,15 +81,19 @@ $ npm run test
 $ npm run test:unit && npm run test:functional
 ```
 
-#### Using a Docker container
+#### Using Docker
 
-Alternatively, for Linux or macOS users, there is a script that builds and runs a Docker container which then executes the test suite. This means no external dependency, apart from a running MySQL server, is needed.
+Alternatively, for Linux or macOS users, there is a script that builds and launches a group of Docker containers as an environment where the automated test suite can execute without any additional dependency (apart from a Docker-based container engine).
 
-The script uses the environment variables described previously and introduces a few new ones. These are mostly meant to be used for configuring the Docker container itself. They allow to specify the path to a Node.js engine image, the network proxy setup and the URL of the NPM registry to use.
+There are three different set of containers. One, built on top of a Node.js Docker image, is responsible to execute the test suite and contains the source tree and all the required 3rd-party dependencies. Another one launches a small DNS proxy with service discovery capabilities that allows to test among other things, DNS SRV connections in a dynamic network. The DNS proxy is also responsible for linking an additional set of multiple containers running MySQL server instances using different versions and/or configuration options. These allow to run specific tests against older versions to check for regressions, ensure compatibility between authentication mechanisms and deprecated server-side authentication plugins, setup TLS certificate chains between client and server, and test multi-host capabilities and connection failover, end-to-end.
+
+The script uses the environment variables described previously and introduces a few new ones. These are mostly meant to be used for configuring the Docker containers. They allow to specify the base Node.js and MySQL Docker images, the network proxy setup, and the URL of the NPM registry to use.
 
 * `BASE_IMAGE` (`container-registry.oracle.com/graalvm/nodejs:latest` by default)
 * `HTTP_PROXY` (value of the environment variable in the host by default)
 * `HTTPS_PROXY` (value of the environment variable in the host by default)
+* `MYSQL_IMAGE` (`mysql/mysql-server` by default)
+* `MYSQL_VERSION` (`latest` by default)
 * `NO_PROXY` (value of the environment variable in the host by default)
 * `NPM_REGISTRY` (`https://registry.npmjs.org/` by default)
 
@@ -98,7 +102,7 @@ There is one additional environment variable called `TEST_PATTERN` which can be
 Ultimately, the script allows an argument which identifies the underlying NPM script that gets executed. So, in theory, any of the available NPM scripts can be executed in the container, but by default, it will execute the `test` script.
 
 ```sh
- # executing the default test script with the default environment (Linux only)
+ # executing the default test script with the default environment
 $ ./test/docker/run.sh
 # executing the unit test suite
 $ ./test/docker/run.sh test:unit
@@ -108,7 +112,18 @@ $ MYSQLX_HOST='<hostname_or_IP_address>' TEST_PATTERN='using CRUD' ./test/docker
 $ MYSQLX_SOCKET='/path/to/socket' ./test/docker/run.sh test:functional
 ```
 
-Similar to when the tests run on a local environment, the `MYSQLX_HOST` variable is only relevant for the functional tests. On Linux, the variable is optional and the Docker container will run using the "host" network mode whilst tests assume the MySQL server is listening on `localhost`. On macOS, since containers run on a virtual machine, host loopback addresses are not reachable. In that case, the `MYSQLX_HOST` variable is required and should specify the hostname or IP address of the MySQL server.
+Users are still able to run the default functional test suite using their own MySQL server instance. Similar to when the tests run on a local environment, the `MYSQLX_HOST` variable is only relevant for the functional tests. The variable is optional and the tests running in the Docker container assume the MySQL server is listening on the host machine and is reachable via `host.docker.internal`. If the MySQL server instance is not running in the host machine, the `MYSQLX_HOST` variable can specify the appropriate hostname or IP address. There is a fallback server instance running in one of the Docker containers which is reachable via `mysql.docker.internal`.
+
+```sh
+# executing the extended functional tests in the Docker environment
+$ ./test/docker/run.sh test:functional:extended
+# executing the default functional tests in the Docker environment
+$ MYSQLX_HOST='mysql.docker.internal' ./test/docker/run.sh test:functional
+# executing the default functional tests with a local MySQL server instance and the extended tests in the Docker environment
+$ ./test/docker/run.sh test:functional:all
+# executing all tests in the Docker environment
+$ MYSQLX_HOST='mysql.docker.internal' ./test/docker/run.sh test:all
+```
 
 Due to some [know limitations](https://github.com/docker/for-mac/issues/483) on the macOS Docker architecture, Unix socket tests can only run on Linux. In that case, if the `MYSQLX_SOCKET` variable is explicitely specified, a shared volume between the host and the container will be created as a mount point from the socket file path in the host and an internal container directory specified as a volume, where the socket file path becomes available.
 
@@ -131,6 +146,23 @@ As a formal rule, a patch should not lead to a decrease of the overall code cove
 
 Besides looking at the content generated via the standard output of your command line, you can also check the report available at `coverage/index.html` using a web browser.
 
+Code coverage reports can also be generated in the Docker environment using the same set of NPM scripts.
+
+```sh
+# generate code coverage reports using the default test suite
+$ ./test/docker/run.sh coverage
+# generate code coverage reports using the unit test suite
+$ ./test/docker/run.sh coverage:unit
+# generate code coverage reports using the default functional test suite
+$ ./test/docker/run.sh coverage:functional
+# generate code coverage reports using the extended functional test suite
+$ ./test/docker/run.sh coverage:functional:extended
+# generate code coverage reports using the full test suite
+$ ./test/docker/run.sh coverage:functional:all
+```
+
+A `coverage` folder will be created in the project root directory containing the artifacts generated by the container, which are copied using a bind mount. If the folder already exists, the contents will be replaced.
+
 ### Code Style and Convention
 
 Any change to the project's source code must also follow the standard set of code style/convention rules enforced by the existing tooling infrastructure. So, before submitting a patch, you can check if it violates any of those rules using the following:

lib/tool/test.js

@@ -47,17 +47,22 @@
   "scripts": {
     "coverage": "nyc npm run test",
     "coverage:functional": "nyc npm run test:functional",
+    "coverage:functional:all": "nyc npm run test:functional:all",
+    "coverage:functional:extended": "nyc npm run test:functional:extended",
     "coverage:summary": "nyc report --reporter=text-summary",
     "coverage:unit": "nyc npm run test:unit",
     "fix:lib": "standardx --fix",
     "fix:types": "eslint --fix -c types/.eslintrc.js .",
     "lint:lib": "standardx --verbose | snazzy",
     "lint:types": "eslint -c types/.eslintrc.js .",
     "lint": "npm run lint:lib && npm run lint:types",
-    "mocha": "mocha --reporter spec --timeout 10000 --recursive",
+    "mocha": "mocha --reporter spec --timeout 30000 --recursive",
     "prepack": "node bin/prepack.js",
     "test": "tsd && npm run mocha test/unit test/functional/default",
+    "test:all": "tsd && npm run mocha test/unit test/functional",
     "test:functional": "npm run mocha test/functional/default",
+    "test:functional:all": "npm mocha test/functional",
+    "test:functional:extended": "npm run mocha test/functional/extended",
     "test:unit": "npm run mocha test/unit",
     "test:types": "tsd"
   },

package.json

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2015, 2021, Oracle and/or its affiliates.
+ * Copyright (c) 2015, 2023, Oracle and/or its affiliates.
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License, version 2.0, as
@@ -35,7 +35,6 @@ module.exports = {
     password: process.env.MYSQLX_PASSWORD || '',
     port: parseInt(process.env.MYSQLX_PORT, 10) || 33060,
     schema: process.env.MYSQLX_DEFAULT_SCHEMA,
-    socket: process.env.MYSQLX_SOCKET,
     tls: { enabled: process.env.MYSQLX_TLS !== 'false' },
     user: process.env.MYSQLX_USER || 'root'
 };

test/config.js

@@ -0,0 +1,31 @@
+# Copyright (c) 2023, Oracle and/or its affiliates.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License, version 2.0, as
+# published by the Free Software Foundation.
+#
+# This program is also distributed with certain software (including
+# but not limited to OpenSSL) that is licensed under separate terms,
+# as designated in a particular file or component or in included license
+# documentation.  The authors of MySQL hereby grant you an
+# additional permission to link the program and your derivative works
+# with the separately licensed software that they have included with
+# MySQL.
+#
+# Without limiting anything contained in the foregoing, this file,
+# which is part of MySQL Connector/Node.js, is also subject to the
+# Universal FOSS Exception, version 1.0, a copy of which can be found at
+# http://oss.oracle.com/licenses/universal-foss-exception.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License, version 2.0, for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin St, Fifth Floor, Boston, MA 02110-1301  USA
+
+services:
+  mysql-connector-nodejs:
+    extra_hosts: []

test/docker/docker-compose.Darwin.yml

@@ -0,0 +1,32 @@
+# Copyright (c) 2023, Oracle and/or its affiliates.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License, version 2.0, as
+# published by the Free Software Foundation.
+#
+# This program is also distributed with certain software (including
+# but not limited to OpenSSL) that is licensed under separate terms,
+# as designated in a particular file or component or in included license
+# documentation.  The authors of MySQL hereby grant you an
+# additional permission to link the program and your derivative works
+# with the separately licensed software that they have included with
+# MySQL.
+#
+# Without limiting anything contained in the foregoing, this file,
+# which is part of MySQL Connector/Node.js, is also subject to the
+# Universal FOSS Exception, version 1.0, a copy of which can be found at
+# http://oss.oracle.com/licenses/universal-foss-exception.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License, version 2.0, for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin St, Fifth Floor, Boston, MA 02110-1301  USA
+
+services:
+  mysql-connector-nodejs:
+    extra_hosts:
+      - "host.docker.internal:host-gateway"

test/docker/docker-compose.Linux.yml

@@ -0,0 +1,37 @@
+# Copyright (c) 2023, Oracle and/or its affiliates.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License, version 2.0, as
+# published by the Free Software Foundation.
+#
+# This program is also distributed with certain software (including
+# but not limited to OpenSSL) that is licensed under separate terms,
+# as designated in a particular file or component or in included license
+# documentation.  The authors of MySQL hereby grant you an
+# additional permission to link the program and your derivative works
+# with the separately licensed software that they have included with
+# MySQL.
+#
+# Without limiting anything contained in the foregoing, this file,
+# which is part of MySQL Connector/Node.js, is also subject to the
+# Universal FOSS Exception, version 1.0, a copy of which can be found at
+# http://oss.oracle.com/licenses/universal-foss-exception.
+#
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License, version 2.0, for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin St, Fifth Floor, Boston, MA 02110-1301  USA
+
+services:
+  mysql-connector-nodejs:
+    volumes:
+      # The test script ensures this configuration is only applied when the
+      # $MYSQLX_SOCKET environment variable is specified.
+      # In that case, it means the tests should use a Unix socket local to the
+      # host machine, which needs to be copied to the container via a bind
+      # mount.
+      - ${MYSQLX_SOCKET}:${MYSQLX_SOCKET}:ro