Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Create standardized Docker development environment via .devcontainer #445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Aug 12, 2021

Conversation

Taytay
Copy link
Contributor

@Taytay Taytay commented Mar 24, 2021

I haven't contributed to SQL.js in a while, and am on a new Machine that doesn't have the EMSDK installed. I remembered it was somewhat fiddly before to get my environment properly configured, and assume I'm not the only one.
There is a relatively new .devcontainer spec championed by VSCode that allows for a standard development environment to be defined by a Dockerfile. VSCode isn't necessary to take advantage of the Docker-based development environment, but using VSCode makes it very straightforward.

I've created a CONTRIBUTING.md that includes instructions for getting started, and also describes the motivations for this change, which are:

  • Allows anyone on ANY platform (Linux, Mac, and Windows) to contribute or compile their own build.
  • It's quicker and easier for any contributor to dive in and fix issues.
  • (Practically) eliminates configuration bugs that are difficult for maintainers to reproduce. Also known as "works on my machine" issues.
  • Allows us to write our scripts assuming that they're always running in a single known environment of a single, known platform.
  • Ensure that all contributors use a known, standardized installation of EMSDK.
  • Allows for a more clearly documented process for updating the EMSDK to a new version.
  • End-Users that simply want to compile and install their own version of SQLite don't have to bother with EMSDK installation in their particular environment.

As you can tell, I think that adopting a standardized development environment would have numerous benefits to our project, and welcome feedback.

@lovasoa
Copy link
Member

lovasoa commented Mar 24, 2021

Oh, this looks very useful, great contribution !

Could you update the github actions to use this Dockerfile ? This would make sure the Dockerfile is tested, and make sure that the CI runs in the same environment as the developer.

@Taytay
Copy link
Contributor Author

Taytay commented Mar 25, 2021

Thanks for the quick response. You bet! Can do.

@Taytay Taytay marked this pull request as draft March 25, 2021 05:33
@Taytay
Copy link
Contributor Author

Taytay commented Mar 25, 2021

Fighting a bit with the Github Actions spec. Will keep banging away at it...

# See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.155.1/containers/typescript-node/.devcontainer/base.Dockerfile
# [Choice] Node.js version: 14, 12, 10
ARG VARIANT="14-buster"
FROM mcr.microsoft.com/vscode/devcontainers/typescript-node:0-${VARIANT}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe using a base of https://hub.docker.com/r/emscripten/emsdk would build faster and be easier ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate it, and I started with that as the base actually! I abandoned it for a couple of reasons.

  1. I thought it might make it harder to rev the version of the SDK we were using, and harder to experiment with versions of the SDK that weren't yet officially Docker-packaged yet. I don't know that this is a compelling goal, but it was part of the reason I abandoned it.
  2. By starting with one of these standard devcontainers published by MS, we get not only node and Typescript (which are easy to install ourselves via the Dockerfile), but some of the other niceties that all of their VSCode Devcontainers get to make them easy to actually develop in, rather than just build in. Zsh, some other config niceties, etc. (https://github.com/microsoft/vscode-dev-containers/blob/master/containers/javascript-node/.devcontainer/base.Dockerfile) It seemed easier to standardize on a devcontainer and install the SDK rather than the other way around. But honestly, if I'm copying lines from Dockerfiles, I could flip it around. It just felt better to install the sdk into the container rather than depend on it as a base.

I will say that I wish Docker files were more composable, and that I could pick and that it was easier to "layer" portions of various public docker files, but I don't expect my "custom" portions of the Dockerfile to change much, other than perhaps the version of node or the emsdk we want to install. If we did want to make this more "composable", another option we have is to use a devcontainer that is one of the standard ones like typescript-node, expose docker to the container itself, and then expect it to be able to shell out to docker to perform the emcc compilation via a standalone emsdk container: docker run -it --rm -v $(pwd):/src --user emscripten --name emsdk emscripten/emsdk:2.0.15 bash) && docker exec -it emsdk make
I didn't like that approach though because I didn't like our development environment requiring/assuming that docker was present. In other words, I didn't want to rewrite our package.json scripts to assume that the docker command existed. As it stands, I've kept the docker thing highly recommended, but still optional. :)

The approach I've checked in worked great for me to standardizing our/my development environment, and installing the emsdk in the Dockerfile is "standard" enough to be obvious if it worked correctly. The two downsides I've seen with my approach after working with Github actions last night:

  1. The container is currently built from scratch on each run which adds to the build time.
  2. I ran into a bug where it couldn't execute the entrypoint.sh file to actually perform the make call. I assume I'm doing something trivially easy to fix. :)

I assume number 2 is easy to fix, so if we desired to fix number 1, we could either check in a built base version of this container to the Github registry and then pull it down as part of the build, or we could look into ways to cache the built docker container between runs.

@lovasoa lovasoa marked this pull request as ready for review August 12, 2021 08:55
@lovasoa lovasoa merged commit e0d35e2 into sql-js:master Aug 12, 2021
@lovasoa
Copy link
Member

lovasoa commented Aug 12, 2021

Ok, sorry I took so much time to get back to it, but here it is !
We have a nice container to work in now.
Hopefully this will lower the barrier to entry in sql.js.

Thanks again @Taytay !

@Taytay
Copy link
Contributor Author

Taytay commented Aug 21, 2021

@lovasoa : No worries! I let it stall out when I tried to get things building in the same container, but after seeing your commit where you got it figured out, I realized I was going about it the wrong way. Nice!
714d44c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.