Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

@BewareMyPower
Copy link
Contributor

Fixes #286

Motivation

The maximum GLIBCXX version on CentOS 7 is 3.4.19. If we want to use the Pulsar Node client on CentOS 7, we have to upgrade the libstdc++.so. The reason is the libstdc++ version is too high for some systems. #285 downgrades the base image to node:18-buster but it still requires the GLIBCXX_3.4.22.

Modifications

  • Use centos:7 as the base image to build Pulsar.node for glibc-based Linux distributions.
  • Add load_test.sh to verify Pulsar.node can be loaded by other docker images and test Node.js 16 and 19 on Debian buster and bullseye.

Documentation

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

tests/docker-load-test.sh Outdated Show resolved Hide resolved
@@ -0,0 +1,42 @@
#!/bin/bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script is only used in Linux-related workflows and is recommended to be placed in the /pkg/linux/ directory

Copy link
Contributor Author

@BewareMyPower BewareMyPower Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

run-unit-tests.sh is also for Linux.

pkg/linux/download-cpp-client.sh

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, But docker-load-test and load-test.sh are only used to test Linux-related in workflows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But pkg/linux directory should represent packaging the Node.js client on Linux. The script here is to test load_test.js on Linux systems.

Assuming there are 3 scripts (e.g. load-test-win.sh, load-test-osx.sh, load-test-linux.sh) that test load_test.js on Windows, macOS and Linux. Do you think it's better to put these scripts under three different directories like pkg/windows, pkg/linux/ and pkg/macos?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, It's good to:

  • Put load_test.js under tests/linux
  • Rename load_test.js with linux_load_test.js

It's bad to put load_test.js under pkg/linux.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot see the difference between "verify" and "test". The only reason I can see for why you want to name it as "verify" is, you do not want to put test scripts under tests/ directory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very clear that:

  • pkg/ directiory contains scripts that package the Node.js binaries in various operation systems.
  • src/ directory containes source code
  • docs/ directory contains documents
  • example/ directory contains examples
  • tests/ directory contains tests

I do not know why do you so insist on distributing tests to different directories.

Some languages tend to put tests under the same directory with the code to test, like Golang, the source code directory contains xxx.go and the associated test xxx_test.go. It seems that your logic looks like that: the tests (load-test.sh) for package scripts should be put under the same directory (pkg/linux/*.sh). However, Node.js is different with those languages. Just like Java, the tests for source code (src/) are put under a separated directory (tests/). If you changed the rule, the for users new to this project, it's not clear to know tests are distributed into how many directories..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to play word games like "verify" and "tests", but since you mentioned this idea, I found a reference here: https://www.quora.com/What-is-the-difference-between-testing-and-verification

Verification: Before a system is developed, there must be a design where the basic requirements for the system are well spelt out. verification is concerned with whether the system is well-engineered, error-free, and so on. Verification will help to determine whether the software is of high quality.
Testing refers to the process of executing an application or program with the intent of detecting potential software bugs. It also has the objective of helping the developers to find out whether the software is working according to the intended purpose.

The load-tests.sh should be executed after the binaries are built and it tests whether the binaries can be executed in a given docker container.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized https://github.com/apache/pulsar-client-python/blob/main/pkg/test-wheel.sh might inspire you to put scripts under the pkg/ directory. I don't think it's a good practice.

First, in this script, /pulsar-client-python/wheelhouse/pulsar_client-*.whl should be the paths inside a docker container. This script cannot be executed locally. It's more like the docker-load-test.sh in this PR.

However, the path of docker-load-test.sh is not related to the Dockerfile under pkg/linux. Instead, the path of /pulsar-client-node/build/Release/pulsar.node and /pulsar-client-node/pulsar-client-node.tar.gz are created by load-test.sh, not the pkg/linux/Dockerfile_linux_glibc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, In short. My point is to verify pulsar.node does not belong to the unit test. Although it is important, it is part of the compilation.

Let's go on and stop tangling.

.github/workflows/ci-pr-validation.yml Outdated Show resolved Hide resolved
pkg/linux/Dockerfile_linux_glibc Show resolved Hide resolved
@BewareMyPower BewareMyPower marked this pull request as draft January 17, 2023 12:54
Fixes apache#286

### Motivation

The maximum GLIBCXX version on CentOS 7 is 3.4.19. If we want to use the
Pulsar Node client on CentOS 7, we have to upgrade the libstdc++.so. The
reason is the libstdc++ version is too high for some systems.
apache#285 downgrades the
base image to `node:18-buster` but it still requires the
`GLIBCXX_3.4.22`.

### Modifications
- Use `centos:7` as the base image to build `Pulsar.node` for
  glibc-based Linux distributions.
- Add `load_test.sh` to verify `Pulsar.node` can be loaded by other
  docker images and test Node.js 16 and 19 on Debian buster and
  bullseye.
@BewareMyPower BewareMyPower force-pushed the bewaremypower/centos-7 branch from 2a2d1ba to b83a5b6 Compare January 18, 2023 14:46
@BewareMyPower BewareMyPower marked this pull request as ready for review January 18, 2023 14:53
tests/load-test.sh Outdated Show resolved Hide resolved
@shibd shibd merged commit cd2a307 into apache:master Feb 10, 2023
shibd pushed a commit to shibd/pulsar-client-node that referenced this pull request Feb 10, 2023
* Make it compatible with GLIBCXX_3.4.19 (CentOS 7)

Fixes apache#286

### Motivation

The maximum GLIBCXX version on CentOS 7 is 3.4.19. If we want to use the
Pulsar Node client on CentOS 7, we have to upgrade the libstdc++.so. The
reason is the libstdc++ version is too high for some systems.
apache#285 downgrades the
base image to `node:18-buster` but it still requires the
`GLIBCXX_3.4.22`.

### Modifications
- Use `centos:7` as the base image to build `Pulsar.node` for
  glibc-based Linux distributions.
- Add `load_test.sh` to verify `Pulsar.node` can be loaded by other
  docker images and test Node.js 16 and 19 on Debian buster and
  bullseye.

* Avoid modifying the Node.js scripts

* Remove the TAR each time the script is executed

* Fix the binary name changed in apache#290

(cherry picked from commit cd2a307)
@BewareMyPower BewareMyPower deleted the bewaremypower/centos-7 branch February 23, 2023 05:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Support CentOS 7

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.